Image processing device and mobile computing device having the same

Patent metadata

Field	Value
Publication number	US-9491442-B2
Application number	US-201514696100-A
Country	US
Kind code	B2
Filing date	Apr 24, 2015
Priority date	Apr 28, 2014
Publication date	Nov 8, 2016
Grant date	Nov 8, 2016

Abstract

In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

First claim

What is claimed is:

1. An image processing device comprising:
a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements;
an image data processing circuit configured to generate image data from pixel signals output from the pixels;
a color data processing circuit configured to,
extract color data from the image data, and
output extracted color data;

a depth data extraction circuit configured to,
extract depth data from the image data, and
output extracted depth data; and

an output control circuit configured to control the output of the color data and the depth data.

2. The image processing device of claim 1 , wherein the image processing device is embodied in one CMOS image sensor chip.

3. The image processing device of claim 1 , wherein the pixel array and the image data processing circuit are integrated into a first chip, and the color data processing circuit, the depth data extraction circuit, and the output control circuit are integrated into a second chip.

4. The image processing device of claim 1 , further comprising:
an application processor configured to control operations of the color data processing circuit, the depth data extraction circuit, and the output control circuit.

5. The image processing device of claim 1 , wherein the image data include long-exposure image data and short-exposure image data.

6. The image processing device of claim 1 , wherein the output control circuit is configured to output the color data and the depth data on a line basis in a parallel manner.

7. The image processing device of claim 1 , wherein the output control circuit is configured to determine an output order of the color data and the depth data.

8. The image processing device of claim 1 , wherein the output control circuit is configured to output the color data and the depth data in a serial manner.

9. The image processing device of claim 8 , wherein the output control circuit is configured to output the color data and the depth data, both having different sizes in a serial manner during one-horizontal time period.

10. The image processing device of claim 8 , wherein
the output control circuit is configured to output the color data corresponding to a first number of lines and the depth data corresponding to a second number lines in a serial manner,
each of the first number of lines and the second number of lines is a natural number, and
the first number of lines is equal to or greater than the second number of lines.

11. The image processing device of claim 8 , wherein
the output control circuit is configured to output the color data corresponding to a first number of pixels and the depth data corresponding to a second number of pixels in a serial manner,
each of the first number of pixels and the second number of pixels are natural numbers, and
the first number of pixels is equal to or greater than the second number of pixels.

12. A mobile computing device comprising:
an image sensor configured to generate image data, the image sensor including,
a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements, and
an image data processing circuit configured to generate the image data from pixel signals output from the pixels; and

a processing circuit configured to process the image data, the processing circuit including,
a color data processing circuit configured to,
extract color data from the image data, and
output extracted color data;

a depth data extraction circuit configured to,
extract depth data from the image data, and
output extracted depth data; and

an output control circuit configured to control the output of the color data and the depth data.

13. The mobile computing device of claim 12 , wherein the image data include long-exposure image data and short-exposure image data, the long-exposure image data corresponding to a first pixel signal output from a first photoelectric conversion element included in each of the plurality of photoelectric conversion elements, and the short-exposure image data corresponding to a second pixel signal output from a second photoelectric conversion element included in each of the plurality of photoelectric conversion elements.

14. An image processing device comprising:
a memory configured to store computer-readable instructions; and
a processor configured to execute the computer-readable instructions to,
generate image data from pixel signals associated with pixels of a pixel array having a plurality of photoelectric conversion elements,
extract color data and depth data from the image data, and
output the color data and the depth data.

15. The image processing device of claim 14 , wherein the pixel array is a phase difference auto-focus pixel array.

16. The image processing device of claim 14 , wherein the image data include long-exposure image data and short-exposure image data, the long-exposure image data corresponding to pixel signals associated with pixels with longer exposure time to light compared to pixel signals associated with the short-exposure image data.

17. The image processing device of claim 16 , wherein
at least a first one of the plurality of photoelectric conversion elements is configured to generate the long-exposure image data, and
at least a second one of the plurality of photoelectric conversation elements is configured to generate the short-exposure image data.

18. The image processing device of claim 16 , wherein the processor is further configured to process the image data prior to extracting the color data using the long exposure image data and the short-exposure image data, the process including at least one of a noise reduction process, edge enhancement process, color correction process and a gamma process.

19. The image processing device of claim 16 , wherein the processor is further configured to process the image data prior to extracting the depth data using the long-exposure image data and the short-exposure image data.

20. The image processing device of claim 19 , wherein the processor is configured to process the image data by extracting disparity data between each pair of the long-exposure image data and the short-exposure image data.

Assignees

• Kim Tae Chan
• Samsung Electronics Co Ltd

Inventors

• Kim Tae Chan

Classifications

• G06T7/50 Primary

  Depth or shape recovery · CPC title

• H04N25/533

  by using differing integration times for different sensor regions · CPC title

• H04N23/12

  with one sensor only · CPC title

• H04N25/76 Primary

  Addressed sensors, e.g. MOS or CMOS sensors · CPC title

• H04N23/73

  by influencing the exposure time · CPC title