Method for generating plural information using camera to sense plural wave bandwidth and apparatus thereof

Patent metadata

Field	Value
Publication number	US-11238279-B2
Application number	US-201916289910-A
Country	US
Kind code	B2
Filing date	Mar 1, 2019
Priority date	Mar 2, 2018
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

According to various embodiments, an electronic device may include a first light emitter comprising first light emitting circuitry configured to output light of a first wave bandwidth, a second light emitter comprising second light emitting circuitry configured to output light of a second wave bandwidth, a camera including an image sensor configured to arrange one or more first pixels for sensing the light corresponding to the first wave bandwidth and the second wave bandwidth and one or more second pixels for sensing the light corresponding to the second wave bandwidth, and a processor. The processor may be configured to control the electronic device to output the first wave bandwidth light using a first light source and the second wave bandwidth light using a second light source, and a control circuit electrically coupled to the image sensor and connected to an application processor through a designated interface. The control circuit may be configured to obtain image data of an external object using the camera from the first wave bandwidth light and the second wave bandwidth light reflected from the external object, to generate first image data of the external object using a difference between first pixel data corresponding to the one or more first pixels and second pixel data corresponding to the one or more second pixels of the image data, to generate second image data of the external object by binning the second pixel data corresponding to the one or more second pixels of the image data, and to transmit the first image data and the second image data to the processor through the designated interface, wherein the processor is configured to generate authentication information based on the first image data and to generate depth information using the second image data.

First claim

What is claimed is:

1. An electronic device comprising:
a first light emitter comprising first light emitting circuitry configured to output first IR (infra-red) light of a first wave bandwidth;
a second light emitter comprising second light emitting circuitry configured to output second IR light of a second wave bandwidth, wherein the second wave bandwidth is different from the first wave bandwidth;
a camera comprising an image sensor comprising a plurality of first pixels for sensing the first IR light corresponding to the first wave bandwidth and the second IR light corresponding to the second wave bandwidth and a plurality of second pixels for sensing the second IR light corresponding to the second wave bandwidth, wherein the plurality of first pixels and the plurality of second pixels are arranged in a grid which alternates each of the first pixels and each of the second pixels; and
a processor electrically coupled to the image sensor and comprising a control circuit connected to an application processor through a designated interface,
wherein the control circuit is configured to:
control the first light emitter to output the first IR light and the second light emitter to output the second IR light;

in response to the first IR light and the second IR light being output, control the image sensor to obtain first pixel data of an external object from the first IR light output from the first light emitter and the second IR light output from the second light emitter, which are reflected from the external object by using the plurality of first pixels of the image sensor,
control to obtain second pixel data of the external object from the second IR light output from the second light emitter, which is reflected from the external object by using the plurality of second pixels of the image sensor,
shift the second pixel data corresponding to the plurality of second pixels of the image sensor by one pixel horizontally or vertically to correspond in position to first pixel data corresponding to the plurality of first pixels of the image sensor,
generate first image data of the external object using a difference between the first pixel data and the shifted second pixel data,
generate second image data of the external object by binning the second pixel data corresponding to the plurality of second pixels of the image data, and
control to transmit the first image data and the second image data to the processor through the designated interface, wherein the processor is configured to generate authentication information based on the first image data and to generate depth information using the second image data.

2. The electronic device of claim 1 , wherein the control circuit is further configured to, for generating the first image data, interpolate values in the first image data corresponding to positions of the plurality of first pixels to positions of the plurality of second pixels.

3. The electronic device of claim 1 , wherein the first wave bandwidth includes 810 nm, and
the second wave bandwidth includes 940 nm.

4. The electronic device of claim 1 , wherein the processor is configured to authenticate the external object using at least one of the authentication information or the depth information.

5. The electronic device of claim 4 , wherein the authentication includes iris authentication.

6. A method for generating an image in an electronic device, the method comprising:
outputting first IR (infra-red) light of a first wave bandwidth by a first light emitter included in the electronic device and second IR light of a second wave bandwidth by a second light emitter included in the electronic device;
in response to the first IR light and the second IR light being output:
obtaining, by using a plurality of first pixels of an image sensor included in the electronic device, first pixel data of an external object from the first IR light and the second IR light reflected from the external object and
obtaining, by using a plurality of second pixels of the image sensor, second pixel data of the external object from the second IR light reflected from the external object, wherein the image sensor comprises the plurality of first pixels for sensing the first IR light corresponding to the first wave bandwidth and the second IR light corresponding to the second wave bandwidth and the plurality of second pixels for sensing the second IR light corresponding to the second wave bandwidth,

wherein the plurality of first pixels and the plurality of second pixels are arranged in a grid which alternates each of the first pixels and each of the second pixels, and
wherein the second wave bandwidth is different than the first wave bandwidth;
shifting the second pixel data corresponding to the plurality of second pixels of the image sensor by one pixel horizontally or vertically to correspond in position to first pixel data corresponding to the plurality of first pixels of the image sensor;
generating first image data of the external object by using a difference between the first pixel data and the shifted second pixel data and by interpolating values in the first pixel data corresponding to positions of the plurality of first pixels to positions of the plurality of second pixels;
generating second image data of the external object by binning the second pixel data corresponding to the plurality of second pixels of the image data; and
transmitting the first image data and the second image data to an application processor through a designated interface, to generate authentication information based on the first image data and to generate depth information using the second image data.

7. The method of claim 6 , wherein generating the first image data further comprises interpolating values in the first image data corresponding to positions of the plurality of first pixels to positions of the plurality of second pixels.

8. The method of claim 6 , wherein the first wave bandwidth includes 810 nm, and
the second wave bandwidth includes 940 nm.

9. The method of claim 6 , further comprising:
authenticating the external object using at least one of the authentication information or the depth information.

10. The method of claim 9 , wherein the authentication includes iris authentication.

11. An electronic device comprising:
a first light emitter comprising first light emitting circuitry configured to output light of a first wave bandwidth;
a second light emitter comprising second light emitting circuitry configured to output light of a second wave bandwidth, wherein the second wave bandwidth is different than the first wave bandwidth;
a camera comprising a pixel array comprising a plurality of first pixels configured to transform the light of the first wave bandwidth and the second wave bandwidth to a first electric signal and a plurality of second pixels configured to transform the light of the second wave bandwidth to a second electric signal, and an image sensor comprising a plurality of filters mounted on at least part of the plurality of second pixels and configured to pass the light of the second wave bandwidth, wherein the plurality of first pixels and the plurality of second pixels are arranged in a grid including a plurality of rows adjacent to each other and a plurality of columns adjacent to each other, the first pixels and the second pixels alternate between the first and second pixels in each row of the plurality of rows and in each column of the plurality of columns; and
a processor,
wherein the processor is configured to control the electronic device to output the light of the first wave bandwidth using a first light source and to output the light of the second wave bandwidth using a second li

Assignees

• Samsung Electronics Co Ltd

Inventors

• Sung Woontahk
• Kang Hwayong
• Kim Jongsun
• Lee Kihuk
• Kim Bongchan

Classifications

• H04N5/2226 Primary

  Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10) · CPC title

• G06V10/141

  Control of illumination · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• G06V40/19

  Sensors therefor · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title