Display control circuit for controlling audio/video and display device including the same

Patent metadata

Field	Value
Publication number	US-12225358-B2
Application number	US-202217974184-A
Country	US
Kind code	B2
Filing date	Oct 26, 2022
Priority date	Dec 14, 2021
Publication date	Feb 11, 2025
Grant date	Feb 11, 2025

Abstract

The present disclosure relates to a technique for selectively reproducing multiple sounds for respective areas of a display through image analysis for each display area. A technique, for individually controlling multiple audio devices by analyzing an image to extract regions of the image and classifying sound types through data learning, can be provided.

First claim

What is claimed is:

1. A display control circuit comprising:
an image analysis circuit configured to analyze features of each region of an input image provided to an audio/video device;
a sound analysis circuit configured to analyze input sound provided to the audio/video device in a frequency domain and a time domain and generate object sound information;
a multi-sound generation circuit configured to generate multi-sound information by matching the object sound information to the features of each region from the image analysis circuit; and
a sound control circuit configured to individually control sound for each area of the audio/video device according to the multi-sound information,
wherein the image analysis circuit is configured to extract keypoints of the image by applying a feature extraction algorithm to an entire region of the input image, and cluster the keypoints and determine local regions comprising clusters to primarily classify objects present in the input image.

2. The display control circuit of claim 1 , wherein the image analysis circuit is configured to set one of the local regions as a first region, perform object analysis on the first region, re-perform object analysis on a second region, located at a predetermined distance based on coordinate information on the first region, in addition to the first region and determine object types.

3. The display control circuit of claim 1 , wherein the sound analysis circuit is configured to divide the input sound into predetermined sections and perform a Fourier transform for extracting frequency components of the input sound.

4. The display control circuit of claim 3 , wherein the sound analysis circuit is configured to input an extracted sound signal in the frequency domain into an amplifier and determine sound features based on an amplified sound signal.

5. The display control circuit of claim 3 , wherein the sound analysis circuit is configured to transform an extracted sound signal in the frequency domain into a signal in the time domain to determine sound features.

6. The display control circuit of claim 3 , wherein the sound analysis circuit is configured to learn extracted sound signals and classify them according to object types.

7. The display control circuit of claim 1 , wherein the multi-sound generation circuit is configured to obtain the multi-sound information by matching position information for each object type acquired by the image analysis circuit and sound information for each object type acquired by the sound analysis circuit.

8. The display control circuit of claim 7 , wherein the sound control circuit is configured to reproduce sound of a first object located in a first area of the audio/video device based on the multi-sound information and stop reproduction of sound of a second object located in a second area.

9. A multi-sound reproduction method comprising:
analyzing features of each region of an input image to an audio/video device using a feature extraction algorithm;
analyzing an input sound to the audio/video device in a frequency domain and a time domain;
generating multi-sound information by matching the input sound to an object for each region of the input image; and
individually controlling a sound signal of each object based on the multi-sound information,
wherein analyzing features of each region of the input image further comprises aligning, in accordance with a time axis, frequency components extracted by performing Fourier transform on the input sound to image the input sound and performing convolutional neural network learning on the imaged input sound.

10. The multi-sound reproduction method of claim 9 , wherein the audio/video device comprises a plurality of exciters for generating vibration of a diaphragm corresponding to a sound signal and the multi-sound information includes position information of the plurality of exciters corresponding to objects.

11. The multi-sound reproduction method of claim 10 , wherein generating the multi-sound information determines a type of input sound for each region corresponding to an object for each region of the input image and matches a position of an exciter to each region to generate the multi-sound information.

12. The multi-sound reproduction method of claim 9 , wherein analyzing features of each region of the input image further comprises performing global analysis for analyzing an entire region of the input image and local analysis for analyzing a region clustered by extracted keypoints.

13. A display device comprising:
a panel configured to display an image;
an exciter disposed on one surface of the panel, the exciter configured to vibrate the panel to generate sound;
a data processing circuit configured to process image data transmitted to the panel; and
a display control circuit configured to process the image data transmitted to the data processing circuit and sound data transmitted to the exciter,
wherein the display control circuit is configured to determine an object by analyzing features of each region of image information provided to the panel, analyze sound information provided to the exciter in a frequency domain and a time domain, and individually control sound for each area of the panel,
wherein the display control circuit is configured to extract keypoints of the image by applying a feature extraction algorithm to an entire region of an input image, and cluster the keypoints and determine local regions comprising clusters to classify objects present in the input image.

14. The display device of claim 13 , wherein the display control circuit is configured to generate multi-sound information by matching the sound information from a sound analysis circuit to features of each region from an image analysis circuit and individually control sound for each area of the panel according to the multi-sound information.

15. The display device of claim 14 , wherein the multi-sound information is generated by combining coordinate information on each object with coordinate information on the exciter.

16. The display device of claim 13 , wherein the data processing circuit is configured to determine an image supply timing of the panel according to the image data provided by the display control circuit and the display control circuit is configured to adjust output of sound for each area according to the image supply timing.

17. The display device of claim 13 , wherein the display control circuit is configured to receive the image information and the sound information provided in real time and change output of the exciter in response to changes in the image information and the sound information.

Assignees

• Lx Semicon Co Ltd

Inventors

• Lee Do Hoon
• Jeon Hyun Kyu
• Lee Ji Won
• Cho Jae Chan

Classifications

• G09G3/002

  to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT · CPC title

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title

• G09G2340/00

  Aspects of display data processing · CPC title

• H04R5/04

  Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)} · CPC title

• G10L25/24

  the extracted parameters being the cepstrum · CPC title