Image processing apparatus, imaging apparatus, and image processing method

Patent metadata

Field	Value
Publication number	US-9715734-B2
Application number	US-201314408056-A
Country	US
Kind code	B2
Filing date	Jun 26, 2013
Priority date	Jun 28, 2012
Publication date	Jul 25, 2017
Grant date	Jul 25, 2017

Abstract

There is provided an image processing apparatus including a distance information generation portion configured to generate first distance information about an object to be measured based on a phase difference between images provided by a plurality of first cameras having a first base length, and generate second distance information about the object to be measured based on a phase difference between images provided by a plurality of second cameras having a second base length that is different from the first base length; and a distance extraction portion configured to extract distance information from an imaging position and the object to be measured based on the first distance information and the second distance information.

First claim

The invention claimed is:

1. An image processing apparatus, comprising:
a distance information generation portion configured to:
generate a first distance information about an object to be measured, wherein the first distance information is generated based on a phase difference between images provided by a plurality of first cameras having a first base length, and
generate a second distance information about the object to be measured, wherein the second distance information is generated based on a phase difference between images provided by a plurality of second cameras having a second base length that is different from the first base length;
a distance extraction portion configured to extract a distance between an imaging position and the object to be measured based on the first distance information and the second distance information, wherein the extracted distance is utilized to process at least one image; and
a refocus coefficient setting portion configured to calculate a refocus coefficient based on a depth of a designated captured image of an image sensor, wherein the designated captured image corresponds to a refocus surface of the object to be measured, wherein the designated captured image is captured by the image sensor based on the refocus coefficient.

2. The image processing apparatus according to claim 1 , wherein the distance extraction portion is further configured to extract the distance to the object to be measured within a mutually overlapped distance range in a first distance range extracted from the first distance information and a second distance range extracted from the second distance information.

3. The image processing apparatus according to claim 1 , further comprising a defect correction part configured to correct a defect attributed to an abnormality of the image sensor.

4. The image processing apparatus according to claim 3 , further comprising a clamp processing part configured to set a black level in pixel data captured by the image sensor after any defect correction by the defect correction part.

5. The image processing apparatus according to claim 1 , further comprising a memory part to store the first and second base lengths.

6. The image processing apparatus according to claim 1 ,
wherein the distance information generation portion is further configured to generate a third distance information about the object to be measured, wherein the third distance information is generated based on a phase difference between images provided by a third camera having a third base length being different from the first base length and the second base length, and
wherein the distance extraction portion is further configured to extract the distance between the imaging position and the object to be measured based on the first, second, and third distance information.

7. An imaging apparatus, comprising:
an image sensor; and
an image processing apparatus, including:
a distance information generation portion configured to:
generate a first distance information about an object to be measured, wherein the first distance information is generated based on a phase difference between images provided by a plurality of first cameras having a first base length; and
generate a second distance information about the object to be measured, wherein the second distance information is generated based on a phase difference between images provided by a plurality of second cameras having a second base length that is different from the first base length;
a distance extraction portion configured to extract a distance between an imaging position of the image sensor and the object to be measured based on the first distance information and the second distance information, wherein the extracted distance is utilized to process at least one image; and
a refocus coefficient setting portion configured to calculate a refocus coefficient based on a depth of a designated captured image of an image sensor, wherein the designated captured image corresponds to a refocus surface of the object to be measured, wherein the designated captured image is captured by the image sensor based on the refocus coefficient.

8. The image apparatus according to claim 7 , wherein each of the first and second cameras include a combination of one microlens of a microlens array and a portion of a light-receiving area of the image sensor, wherein each microlens corresponds to a different light-receiving area of the image sensor.

9. The image apparatus according to claim 8 , further comprising a distance information providing part including:
a distance information calculation portion including the distance information generation portion and the distance extraction portion; and
a phase difference detection portion configured to generate a phase difference map based on captured image data of the image sensor.

10. The image apparatus according to claim 9 , further comprising a sort processing part
configured to process the captured image data based on the refocus coefficient.

11. The image apparatus according to claim 10 , further comprising a clamp processing part configured to set a black level in pixel data captured by the image sensor, wherein a determined processing of the captured data provided by the clamp processing part is implemented based on the refocus coefficient.

12. The image processing apparatus according to claim 9 , wherein the refocus coefficient setting portion is further configured to calculate the refocus coefficient, from the distance from the imaging position to the refocus surface of the object to be measured, based on the phase difference map.

13. The image apparatus according to claim 9 ,
wherein each light-receiving area of the image sensor corresponds to a different unit image, and each unit image includes an array of pixels, and
wherein the phase difference detection portion is configured to generate a plurality of disparity images based on the captured image data, wherein pixel data from a given pixel within the array of pixels for each unit image is extracted and combined to generate each disparity image for detection of the phase differences, wherein the given pixels are at the same relative position within the array of pixels for the each unit image.

14. The image apparatus according to claim 13 ,
wherein a number of the plurality of disparity images generated are equal to a number of pixels in each unit image, and
wherein the phase difference map is generated based on the plurality of disparity images and indicates phase differences per unit area.

15. The image apparatus according to claim 7 , wherein the distance extraction portion is further configured to extract the distance to the object to be measured within a mutually overlapped distance range in a first distance range extracted from the first distance information and a second distance range extracted from the second distance information.

16. The image apparatus according to claim 7 , further comprising a defect correction part configured to correct a defect attributed to an abnormality of the image sensor.

17. The image apparatus according to claim 16 , further comprising a clamp processing part configured to set a black level in pixel data of the captured data of the image sensor after any defect correction by the defect correction part.

18. The imaging apparatus according to claim 7 , further comprising a memory part to store the first and second base lengths.

19. The imaging apparatus according to claim 7 ,
wherein the distance information generation portion is further

Assignees

• Sony Corp

Inventors

• Hayasaka Kengo

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T7/557

  from light fields, e.g. from plenoptic cameras · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06T2207/20228

  Disparity calculation for image-based rendering · CPC title

• G06T7/0065 Primary

  Physics · mapped topic