Display control device, method, and program

Patent metadata

Field	Value
Publication number	US-9817485-B2
Application number	US-201214116387-A
Country	US
Kind code	B2
Filing date	May 9, 2012
Priority date	May 17, 2011
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

The present technique relates to a display control device, a method, and a program that can improve user's operability of a free-cursor type user interface.
An imaging unit outputs an RGB image based on a pixel signal corresponding to a visible light among light input through a lens and an IR image based on a pixel signal corresponding to an infrared light among light input through the lens. An ROI set unit sets a movement area of an infrared light source in the IR image based on the RGB image, and a display control unit controls a cursor movement amount on a display unit according to movement of the infrared light source in the movement area. The present technique can be applied to a television set that monitors a viewing condition of a viewer, for example.

First claim

The invention claimed is:

1. A device comprising:
a display unit on which can be displayed a menu of operations;
a bandpass filter that bandpasses visible light within a range of visible wavelengths and infrared light within a range of infrared light wavelengths;
a color filter array;
an image sensor to receive the visible light and the infrared light after it has passed through the bandpass filter and the color filter array and output a visible light image based the visible light and an infrared light image based on the infrared light, the imaging sensor having pixels that photoconvert the visible light into the visible light image and pixels that photoconvert the infrared light into the infrared light image;
a visible image processing unit that processes the visible light image from the imaging unit image sensor;
an infrared image processing unit that processes the infrared light image from the image sensor;
a bright spot extraction unit configured to determine a location of a bright spot within the infrared light image based on information from the infrared image processing unit;
an area set unit configured to set a movement area of the bright spot in the infrared light image based on visible light information from the visible image processing unit; and
a display control unit that controls the display unit based, at least in part, on information from the area set unit to control a cursor movement amount on a display unit according to movement of the bright spot in the movement area,
wherein,
the image sensor is controlled to image the visible light or the infrared light at different times and for successive frames,
the area set unit is configured to determine if the bright spot is within a previously determined movement area for a prior frame and, (a) if it is not, to generate a larger movement area and, and (b) if it is, to determine if the bright spot has moved locations, and if the bright spot has moved locations, to generate a movement area for a succeeding frame with the bright spot positioned at the center of the movement area for the succeeding frame, and
the display control unit is configured to move a cursor on the menu of operations according to movement of the bright spot.

2. The device according to claim 1 , wherein the area set unit sets a size of the movement area of the bright spot in the infrared light image based on a subject in the visible light image and makes a position of the bright spot to be always at the center of the movement area in the infrared light image.

3. The device according to claim 2 , further comprising a face detection unit configured to detect a face of the subject in the visible light image, wherein the area set unit sets the size of the movement area of the bright spot in the infrared light image based on a size of the face of the subject in the visible light image.

4. The device according to claim 3 , further comprising a specifying unit configured to specify one of a plurality of subjects based on positions of the faces of the subjects in the visible light image and the position of the bright spot of the infrared light image when more than one faces are detected by the face detection unit, wherein the area set unit sets the size of the movement area of the bright spot in the infrared light image based on the size of the face of the subject specified by the specifying unit.

5. The device according to claim 2 , further comprising a distance calculation unit configured to calculate a distance from the subject based on the visible light image, wherein the area set unit sets the size of the movement area of the bright spot in the infrared light image based on the distance from the subject.

6. The device according to claim 5 , wherein the area set unit sets the size of the movement area of the bright spot in the infrared light image based on the distance from the subject and a size of the display unit.

7. The device according to claim 1 , wherein the display control unit is configured to switch parameters related to imaging by the image sensor so as to be most suitable to the visible light image and the infrared light image respectively, which are output by the image sensor.

8. The device according to claim 1 , wherein the display control unit is configured to set a parameter related to imaging by the image sensor so as to be most suitable to the visible light image output by the image sensor.

9. The device according to claim 8 , further comprising a difference image generation unit configured to generate a difference image based on infrared light images in different time frames, wherein the area set unit sets the movement area of the bright spot which makes movement in the difference image.

10. The device according to claim 1 , further comprising a distance calculation unit configured to calculate a distance from a subject based on the visible light image, wherein the display control unit displays a target to be displayed on the display unit in a size corresponding to the distance from the subject and moves a cursor on the display unit according to movement of the bright spot in the movement area.

11. The device according to claim 1 , wherein a pixel signal output as the infrared light image from the image sensor is output from an imaging element in which a filter that transmits the infrared light is placed at one unit when four pixels of RGB pixels are assumed as one pixel unit.

12. The device according to claim 11 , wherein the pixel signal output as the infrared light image from the image sensor is output from the imaging element in which the filter that transmits the infrared light is placed at one of G pixels in a Bayer array.

13. The device according to claim 1 , wherein a pixel signal output as the infrared light image from the image sensor is output from an imaging element in which a filter that transmits the infrared light is placed at one unit when four pixels of white pixels are assumed as one unit.

14. The device according to claim 1 , wherein:
a wavelength of the infrared light of the bright spot is different from a wavelength of an infrared light emitted from an LED (Light Emitting Diode) of a remote controller for operating the device, and
the image sensor has a filter having a transmission band for the wavelength of the infrared light of the bright spot.

15. The device of claim 1 , wherein the bandpass filter only passes light having wavelengths of 400 nm to 650 nm and 940 nm.

16. A method of controlling a display device comprising (a) a display unit on which can be displayed a menu of operations; (b) a bandpass filter that bandpasses visible light within a range of visible wavelengths and infrared light within a range of infrared light wavelengths; (b) a color filter array; (c) an image sensor to receive the visible light and the infrared light after it has passed through the bandpass filter and the color filter array and output a visible light image based the visible light and an infrared light image based on the infrared light, the imaging sensor having pixels that photoconvert the visible light into the visible light image and pixels that photoconvert the infrared light not the infrared light image; (d) a visible image processing unit that processes the visible light image from the image sensor; (e) an infrared image processing unit that processes the infrared light image from the image sensor; (f) a bright spot extraction unit configured to determine a location of a bright spot within the infrared light image based on information from the infrared image processing unit; (g) an area set unit configured to set a movement area of the bri

Assignees

• Ozawa Ken
• Iwasaki Masanori
• Mogi Hideaki
• Sony Semiconductor Solutions Corp

Inventors

• Ozawa Ken
• Iwasaki Masanori
• Mogi Hideaki

Classifications

• G06F3/0304 Primary

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30196

  Human being; Person · CPC title