Image signal processor for generating depth map from phase detection pixels and device having the same

Patent metadata

Field	Value
Publication number	US-2016267666-A1
Application number	US-201615063109-A
Country	US
Kind code	A1
Filing date	Mar 7, 2016
Priority date	Mar 9, 2015
Publication date	Sep 15, 2016
Grant date	—

Abstract

An image signal processor including a CPU is provided. The CPU receives image data and positional information of phase detection pixels from an imaging device, extracts first phase detection pixel data and second phase detection pixel data from the image data using the positional information of phase detection pixels, computes first phase graphs from the first phase detection pixel data based upon moving a first window, computes second phase graphs from the second phase detection pixel data based upon moving a second window, computes disparities of the phase detection pixels using the first phase graphs and the second phase graphs, and generates a depth map using the disparities.

First claim

What is claimed is:

1 . An image signal processor comprising:
a central processing unit (CPU) configured to receive image data and positional information of phase detection pixels from an imaging device, extract first phase detection pixel data and second phase detection pixel data from the image data using the positional information of the phase detection pixels, compute first phase graphs from the first phase detection pixel data based upon moving a first window, compute second phase graphs from the second phase detection pixel data based upon moving a second window, compute disparities of the phase detection pixels using the first phase graphs and the second phase graphs, and generate a depth map based upon the disparities.

2 . The image signal processor of claim 1 , wherein the CPU is configured to convert the disparities into gray levels, and generate the depth map based upon the gray levels.

3 . The image signal processor of claim 1 , wherein the CPU is configured to set the first window and the second window having a same size as each other, and compute each of the first phase graphs and each of the second phase graphs based upon moving the first window and the second window in a scanning direction.

4 . The image signal processor of claim 1 , wherein each size of the first phase detection pixel data and the second phase detection pixel data is greater than a size of the depth map.

5 . The image signal processor of claim 1 , wherein the CPU is configured to generate three-dimensional image data using color data included in the image data and the depth map.

6 . The image signal processor of claim 1 , wherein the CPU is configured to recognize a gesture of a user corresponding to the image data using the depth map.

7 . A mobile computing device comprising:
an imaging device including color pixels and phase detection pixels; and an application processor including an image signal processor having a first CPU, the application processor including a second CPU; wherein at least one of the first CPU and the second CPU is configured to receive image data and positional information of the phase detection pixels from the imaging device, extract first phase detection pixel data and second phase detection pixel data from the image data using the positional information of phase detection pixels, compute first phase graphs from the first phase detection pixel data based upon moving a first window, compute second phase graphs from the second phase detection pixel data based upon moving a second window, compute disparities of the phase detection pixels using the first phase graphs and the second phase graphs, and generate a depth map based upon the disparities.

8 . The mobile computing device of claim 7 , further comprising a memory configured to store the image data and the positional information of the phase detection pixels.

9 . The mobile computing device of claim 7 , wherein the at least one of the first CPU and the second CPU is configured to convert the disparities into gray levels, and generate the depth map based upon the gray levels.

10 . The mobile computing device of claim 7 , wherein the application processor further includes a graphics processing unit (GPU), and the at least one of the first CPU and the second CPU is configured to generate three-dimensional image data using color data included in the image data and the depth map, and the GPU is configured to render the three-dimensional image data.

11 . The mobile computing device of claim 7 , wherein the at least one of the first CPU and the second CPU is configured to determine a gesture of a user corresponding to the image data using the depth map.

12 . The mobile computing device of claim 7 , wherein the at least one of the first CPU and the second CPU is configured to generate a control signal for auto-focus on a region of interest (ROI) using the depth map, and transmit the control signal to the imaging device.

13 . The mobile computing device of claim 7 , wherein the at least one of the first CPU and the second CPU is configured to set the first window and the second window having a same size each other, and generate each of the first phase graphs and each of the second phase graphs based upon moving the first window and the second window in a scanning direction at a same time.

14 . The mobile computing device of claim 7 , wherein each size of the first phase detection pixel data and the second phase detection pixel data is greater than a size of the depth map.

15 . An image processing system comprising:
an imaging device including color pixels and phase detection pixels; a memory configured to store image data and positional information of the phase detection pixels; a controller including electronic circuitry configured to generate first phase detection pixel data and second phase detection pixel data from the image data using the positional information of the phase detection pixels, generate first phase representations from the first phase detection pixel data based upon moving a first window, generate second phase representations from the second phase detection pixel data based upon moving a second window, determine disparities of the phase detection pixels using the first phase representations and the second phase representations, and generate a depth map based upon the disparities; and a display configured to display images from the imaging device based upon the controller.

16 . The image processing system of claim 15 , wherein the electronic circuitry is configured to convert the disparities into gray levels, and generate the depth map based upon the gray levels.

17 . The image processing system of claim 15 , wherein the electronic circuitry is configured to set the first window and the second window having a same size as each other, and compute each of the first phase representations and each of the second phase representations based upon moving the first window and the second window in a scanning direction.

18 . The image processing system of claim 15 , wherein each size of the first phase detection pixel data and the second phase detection pixel data is greater than a size of the depth map.

19 . The image processing system of claim 15 , wherein the electronic circuitry is configured to generate three-dimensional image data using color data included in the image data and the depth map.

20 . The image processing system of claim 15 , wherein the electronic circuitry is configured to recognize a gesture of a user corresponding to the image data using the depth map.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Donghoon
• Cho Ju Hyun

Classifications

• H04N25/704

  Pixels specially adapted for focusing, e.g. phase difference pixel sets · CPC title

• G06T7/557 Primary

  from light fields, e.g. from plenoptic cameras · CPC title

• H04N23/672

  based on the phase difference signals · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title