Image processing device configured to generate depth map and method of operating the same

Patent metadata

Field	Value
Publication number	US-10929994-B2
Application number	US-201715833518-A
Country	US
Kind code	B2
Filing date	Dec 6, 2017
Priority date	Dec 7, 2016
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

Provided is an image processing device. The device includes an active pixel sensor array including a plurality of pixels configured to generate a plurality of signals corresponding to a target, and an image processor configured to generate a depth map about the target based on an intensity difference of two signals among the plurality of signals.

First claim

What is claimed is:

1. An image processing device comprising:
an active pixel sensor array including a plurality of pixels configured to generate a plurality of signals corresponding to a target; and
an image processor configured to generate a depth map about the target based on an intensity difference between at least two signals among the plurality of signals, the at least two signals respectively corresponding to at least two adjacent pixels among the plurality of pixels;
wherein the depth map is generated without a light source other than natural light;
wherein the active pixel sensor array comprises a first pixel to a third pixel disposed in a first row among the plurality of pixels,
wherein the image processor is further configured to generate the depth map based on:
a first difference between a first signal corresponding to the first pixel and a second signal corresponding to the second pixel; and
a second difference between the second signal corresponding to the second pixel and a third signal corresponding to the third pixel;

wherein the active pixel sensor array further comprises a fourth pixel to a sixth pixel disposed in a second row among the plurality of pixels, and
wherein the image processor is further configured to generate the depth map based on:
a third difference between a fourth signal corresponding to the fourth pixel and a fifth signal corresponding to the fifth pixel; and
a fourth difference between the fifth signal corresponding to the fifth pixel and a sixth signal corresponding to the sixth pixel.

2. The image processing device of claim 1 , wherein the processor reflects a difference between the first signal and the fourth signal when calculating the third difference and the fourth difference.

3. The image processing device of claim 1 , wherein the active pixel sensor array comprises the first pixel to a fourth pixel disposed in the first row among the plurality of pixels,
wherein the image processor is further configured to generate the depth map based on:
a fifth difference between the first signal corresponding to the first pixel and the third signal corresponding to the third pixel; and
a sixth difference between the second signal corresponding to the second pixel and a seventh signal corresponding to the fourth pixel disposed in the first row.

4. The image processing device of claim 1 , wherein the active pixel sensor array comprises the first pixel to a fifth pixel disposed in the first row among the plurality of pixels,
wherein the image processor is further configured to generate the depth map based on:
a fifth difference between the first signal corresponding to the first pixel and the third signal corresponding to the third pixel; and
a sixth difference between the third signal corresponding to the third pixel and a seventh signal corresponding to the fifth pixel disposed in the first row.

5. An image processing device comprising:
an active pixel sensor array including a plurality of pixels configured to receive optical signals reflected from a target and convert the optical signals into electrical signals;
a digital signals converter configured to process the electrical signals to generate intensity values of the electrical signals;
a memory configured to store the processed electrical signals and the intensity values of the electrical signals; and
an image processor configured to calculate a difference between two intensity values corresponding respectively to at least two adjacent pixels among the plurality of pixels, selected from the intensity values, and generate a depth map about the target based on the difference, without a light source other than natural light;
wherein the active pixel sensor array comprises first to third pixels disposed in a first row among the plurality of pixels,
wherein the first to third pixels correspond to first to third intensity values of the electrical signals, respectively, and
wherein the image processor is further configured to generate the depth map based on:
a first difference between the first intensity value and the second intensity value; and
a second difference between the second intensity value and the third intensity value; and

wherein the active pixel sensor array further comprises fourth pixel to sixth pixels disposed in a second row among the plurality of pixels,
wherein the fourth to sixth pixels correspond to fourth to sixth intensity values of the electrical signals, respectively, and
wherein the image processor is further configured to generate the depth map based on:
a third difference between the fourth intensity value and the fifth; and
a fourth difference between the fifth intensity value and the sixth intensity value.

6. The image processing device of claim 1 , further comprising a digital signal converter configured to convert the signals to digital signals.

7. The image processing device of claim 1 , wherein each of the plurality of pixels comprises a CMOS based sensor.

8. A method of generating a depth map using an image sensor including an active pixel sensor array with a plurality of pixels, the method comprising:
outputting a plurality of signals corresponding to a target from the plurality of pixels;
generating depth map profile data based on an intensity difference between at least two selected signals among the plurality of signals, the at least two signals respectively corresponding to at least two adjacent pixels among the plurality of pixels; and
generating a depth map about the target based on the depth map profile data without a light source other than natural light;
wherein the active pixel sensor array comprises a first pixel to a third pixel disposed in a first row among the plurality of pixels,
wherein the generating of the depth map profile data comprises:
calculating a first difference between a first signal corresponding to the first pixel and a second signal corresponding to the second pixel;
calculating a second difference between the second signal corresponding to the second pixel and a third signal corresponding to the third pixel; and
generating the depth map based on the first difference and the second difference;
wherein the active pixel sensor array further comprises a fourth pixel to a sixth pixel disposed in a second row among the plurality of pixels,
wherein the generating of the depth map profile data further comprises:
calculating a third difference between a fourth signal corresponding to the fourth pixel and a fifth signal corresponding to the fifth pixel;
calculating a fourth difference between the fifth signal corresponding to the fifth pixel and a sixth signal corresponding to the sixth pixel; and
generating the depth map based on the third difference and the fourth difference.

9. The method of claim 8 , wherein the calculating of the third difference and the fourth difference is executed in consideration of a difference of the first signal and the fourth signal.

10. The method of claim 8 , wherein the active pixel sensor array comprises the first pixel to a fourth pixel disposed in the first row among the plurality of pixels,
wherein the generating of the depth map profile data further comprises:
calculating a fifth difference between the first signal corresponding to the first pixel and the third signal corresponding to the third pixel;
calculating a sixth difference between the second signal corresponding to the second pixel and a seventh signal corresponding to the fourth pixel disposed in the first row; and
generating the depth map based on the fifth difference and the sixth difference.

11. The method of claim 8 , wherein the active pixel sensor array comprises the first pixel to a fif

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Yu Hyun Kyu
• Kang Sung Weon
• Koo Bon Tae
• Lee Joo Hyun

Classifications

• H04N25/76

  Addressed sensors, e.g. MOS or CMOS sensors · CPC title

• H04N13/218

  using spatial multiplexing · CPC title

• H04N13/128

  Adjusting depth or disparity · CPC title

• H04N13/271

  wherein the generated image signals comprise depth maps or disparity maps · CPC title

• G06T7/50 Primary

  Depth or shape recovery · CPC title