Image processing apparatus and image processing method

Patent metadata

Field	Value
Publication number	US-10134136-B2
Application number	US-201314427403-A
Country	US
Kind code	B2
Filing date	Aug 22, 2013
Priority date	Oct 24, 2012
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

According to an embodiment of the present technology, there is provided an image processing apparatus including a parallax information generation unit. The parallax information generation unit generates parallax information based on a first phase difference distribution and a second phase difference distribution, the first phase difference distribution being generated on a pixel by pixel basis for the first parallax image and the second parallax image, the second phase difference distribution being generated on a sub-pixel by sub-pixel basis based on the first phase difference distribution.

First claim

The invention claimed is:

1. An image processing apparatus, comprising:
at least one processor configured to:
acquire a first parallax image and a second parallax image, wherein a viewpoint of the first parallax image is different from a viewpoint of the second parallax image;
generate a first phase difference distribution on a pixel by pixel basis based on a comparison of a first unit area of the first parallax image with a second unit area of the second parallax image;
generate a first luminance distribution of a first detection area in the first parallax image, wherein the first detection area comprises a first edge portion of the first parallax image;
generate a second luminance distribution of a second detection area in the second parallax image, wherein the second detection area comprises a second edge portion of the second parallax image;
determine a first edge angle of the first edge portion based on the first luminance distribution;
determine a second edge angle of the second edge portion based on the second luminance distribution;
determine a third luminance distribution for evaluation of the first parallax image based on the first edge angle;
determine a fourth luminance distribution for evaluation of the second parallax image based on the second edge angle;
determine a difference between first coordinates of a first pixel in the first detection area and second coordinates of a second pixel in the second detection area, based on the third luminance distribution and the fourth luminance distribution,
wherein the first pixel and the second pixel have a same luminance, and wherein the difference between the first coordinates and the second coordinates corresponds to a phase difference on a sub-pixel by sub-pixel basis;
generate a second phase difference distribution based on the difference between the first coordinates and the second coordinates; and
generate parallax information based on the first phase difference distribution and the second phase difference distribution.

2. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to correct the first phase difference distribution based on edge information detected from the first phase difference distribution to generate the second phase difference distribution.

3. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to calculate a correlation value between a fifth luminance distribution in a first pixel group and a sixth luminance distribution in a second pixel group to generate the second phase difference distribution, and wherein the first pixel group constitutes the first parallax image, and the second pixel group constitutes the second parallax image.

4. The image processing apparatus according to claim 3 , wherein the at least one processor is further configured to calculate the correlation value based on a luminance difference between a third pixel of a plurality of pixels and a fourth pixel of the plurality of pixels, wherein the fourth pixel is consecutive to the third pixel, wherein the plurality of pixels comprise the first pixel group and the second pixel group, and wherein the first pixel group is superimposed on the second pixel group on the pixel by pixel basis.

5. The image processing apparatus according to claim 4 , wherein the at least one processor is further configured to calculate the correlation value based on a length of a broken line, wherein the broken line connects luminance values of the plurality of pixels.

6. The image processing apparatus according to claim 4 , wherein the at least one processor is further configured to calculate the correlation value based on a surface area of a three-dimensional surface, wherein the three-dimensional surface connects luminance values of the plurality of pixels.

7. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to generate distance information as the parallax information, wherein the distance information indicates a distance between a measured object and an imaging unit.

8. The image processing apparatus according to claim 7 , wherein the at least one processor is further configured to generate a refocused image based on the generated distance information.

9. The image processing apparatus according to claim 1 , further comprising an imaging unit configured to acquire a plurality of parallax images, wherein the plurality of parallax images comprise the first parallax image and the second parallax image.

10. The image processing apparatus according to claim 1 , wherein the sub-pixel is a pixel unit smaller than one pixel.

11. The image processing apparatus according to claim 1 , wherein the at least one processor is further configured to:
measure a first luminance of each pixel of the first detection area in the first parallax image;
measure a second luminance of each pixel of the second detection area in the second parallax image;
generate the first luminance distribution based on the first luminance of each pixel of the first detection area; and
generate the second luminance distribution based on the second luminance of each pixel of the second detection area.

12. An image processing method, comprising:
acquiring a first parallax image and a second parallax image, wherein a viewpoint of the first parallax image is different from a viewpoint of the second parallax image;
generating a first phase difference distribution on a pixel by pixel basis based on a comparison of a first unit area of the first parallax image with a second unit area of the second parallax image;
generating a first luminance distribution of a first detection area in the first parallax image, wherein the first detection area comprises a first edge portion of the first parallax image;
generating a second luminance distribution of a second detection area in the second parallax image, wherein the second detection area comprises a second edge portion of the second parallax image;
determining a first edge angle of the first edge portion based on the first luminance distribution;
determining a second edge angle of the second edge portion based on the second luminance distribution;
determining a third luminance distribution for evaluation of the first parallax image based on the first edge angle;
determining a fourth luminance distribution for evaluation of the second parallax image based on the second edge angle;
determining a difference between first coordinates of a first pixel of the first detection area and second coordinates of a second pixel of the second detection area, based on the third luminance distribution and the fourth luminance distribution,
wherein the first pixel and the second pixel have a same luminance, and wherein the difference between the first coordinates and the second coordinates corresponds to a phase difference on a sub-pixel by sub-pixel basis;
generating a second phase difference distribution based on the difference between the first coordinates and the second coordinates; and
generating parallax information based on the first phase difference distribution and the second phase difference distribution.

Assignees

• Sony Corp

Inventors

• Hayasaka Kengo
• Ito Katsuhisa

Classifications

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G01S11/12

  using electromagnetic waves other than radio waves · CPC title

• G01C3/06

  Use of electric means to obtain final indication · CPC title