Image pickup apparatus, depth information acquisition method and program

Patent metadata

Field	Value
Publication number	US-9762788-B2
Application number	US-201314406866-A
Country	US
Kind code	B2
Filing date	Jul 23, 2013
Priority date	Jul 31, 2012
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

An image pickup apparatus includes an imaging unit that acquires a first image by capturing an object at a first focusing position, and acquires a second image after acquiring the first image, by capturing the object at a second focusing position different from the first. The second image has different blur than the first image. A depth information determination unit determines depth information of the object on the basis of a difference in blur between the first image and the second image. A focusing position setting unit sets a focus movement amount that is a difference between the first focusing position and the second focusing position, in accordance with an F-number of the imaging unit. The second focusing position is set such that an absolute value of the focus movement amount is larger when the F-number is large, as compared to when the F-number is relatively small.

First claim

The invention claimed is:

1. An image pickup apparatus comprising:
an imaging unit acquiring a first image by capturing an object at a first focusing position, and acquiring a second image after acquiring the first image, by capturing the object at a second focusing position that is different from the first focusing position, the second image having different blur than that of the first image;
a depth information determination unit determining depth information of the object on a basis of a difference in blur between the first image and the second image; and
a focusing position setting unit setting a focus movement amount that is a difference between the first focusing position and the second focusing position, in accordance with an F-number of the imaging unit,
wherein, in a case that the F-number of the image unit is changed automatically or by a user operation to a changed F-number, the focusing position setting unit sets the second focusing position in accordance with the changed F-number in such a manner that an absolute value of the focus movement amount is greater when the F-number is large, as compared to when the F-number is relatively small, and
the focusing position setting unit sets the second focusing position in such a manner that the focus movement amount satisfies the formula given below:
d=kF 2 λ, −16≦ k≦ 16, k≠ 0,

where d is the focus movement amount at an image plane, k is a coefficient, F is the F-number, and λ is a wavelength of light used to capture the first image and the second image.

2. The image pickup apparatus according to claim 1 , wherein the depth information determination unit determines the depth information on a basis of a ratio between a peak value of a point speed function (PSF) of the imaging unit obtained from the first image and a peak value of a PSF obtained from the second image.

3. The image pickup apparatus according to claim 2 ,
wherein the peak value of the PSF of the imaging unit is a maximum at a focusing position and decreases as defocusing occurs, reaching a local minimum value when an amount of defocus from the focusing position is a prescribed amount, and
an absolute value of the focus movement amount is set to a value less than the prescribed amount.

4. The image pickup apparatus according to claim 1 , wherein the depth information determination unit determines the depth information on a basis of a correlation between a point speed function (PSF) of the imaging unit obtained from the first image and a PSF of the imaging unit obtained from the second image.

5. The image pickup apparatus according to claim 4 ,
wherein a variance value of the PSF of the imaging unit is a maximum at a focusing position and decreases as defocusing occurs, reaching a value 1/e 2 of the maximum value when an amount of defocus from the focusing position is a prescribed amount, and
an absolute value of the focus movement amount is set to a value that is less than two times the prescribed amount.

6. The image pickup apparatus according to claim 1 , wherein, if the depth information determination unit is a unit that determines whether or not the object is included in one particular depth range, the focus movement amount is set so as to satisfy the formula given below:
d=kF 2 λ, −16≦ k<− 8, 8< k≦ 16.

7. The image pickup apparatus according to claim 1 , wherein, if the depth information determination unit is a unit that determines in which of three or more depth ranges the object is included, the focus movement amount is set so as to satisfy the formula given below:
d=kF 2 λ, −8≦ k< 0, 0< k≦ 8.

8. The image pickup apparatus according to claim 7 , wherein the focus movement amount is set so as to satisfy the formula given below:
d=kF 2 λ, −4< k≦− 2, 2≦ k< 4.

9. The image pickup apparatus according to claim 1 , wherein, if the depth information determination unit determines depth information of an object that is situated to the side of infinity from the first focusing position, then the second focusing position is set to the side of infinity from the first focusing position.

10. The image pickup apparatus according to claim 1 , wherein, if the depth information determination unit determines depth information of an object that is situated to the side of the image pickup apparatus from the first focusing position, then the second focusing position is set to the side of the image pickup apparatus from the first focusing position.

11. The image pickup apparatus according to claim 1 , wherein the imaging unit has an image sensor equipped with a color filter having an arrangement of RGB pixels, and λ is a central wavelength of a color filter of a green pixel.

12. The image pickup apparatus according to claim 1 , wherein λ is 587.56 nm.

13. The image pickup apparatus according to claim 1 , wherein the focusing position setting unit has a memory storing a correspondence table of F-numbers and focus movement amounts, and reads out the focus movement amount from the correspondence table in accordance with the changed F-number.

14. The image pickup apparatus according to claim 1 , wherein the second focusing position is automatically changed in accordance with the changed F-number in such a manner that an absolute value of the focus movement amount is proportional to a square of the F-number.

15. The image pickup apparatus according to claim 1 , wherein the depth information determination unit determines the depth information on a basis of a correlation between a partial region of the first image and a corresponding partial region of the second image.

16. A depth information acquisition method comprising:
a focusing position setting step of setting a focus movement amount that is a difference between a first focusing position and a second focusing position, in accordance with an F-number in an imaging unit, the second focusing position being different from the first focusing position;
a first imaging step of acquiring a first image by capturing an object at a first focusing position;
a second imaging step of acquiring a second image having different blur from that of the first image, by capturing the object at the second focusing position, after the first imaging step; and
a depth information determining step of determining depth information of the object on a basis of a difference in blur between the first image and the second image,
wherein, in a case that the F-number of the imaging unit is changed automatically or by a user operation to a changed F-number, the second focusing position is automatically changed in accordance with the changed F-number in such a manner that an absolute value of the focus movement amount is greater when the F-number is large, as compared to when the F-number is relatively small, and
the second focusing position is set in such a manner that the focus movement amount satisfies the formula given below:
d=kF 2 λ, −16≦ k≦ 16, k≠ 0,

where d is the focus movement amount at an image plane, k is a coefficient, F is the F-number, and λ is a wavelength of light used to capture the first image and the second image.

17. A non-transitory computer readable storage medium storing a program that causes a computer to execute an acquisition method comprising:
a focusing position setting step of setting a focus movement amount that is a difference between a first focusing position and a second focusing position, in accordance with an F-number of an imaging unit, the second focusing position being different from the first focusing position;
a first imaging step of acquiring a first image by cap

Assignees

• Canon Kk

Inventors

• Komatsu Satoru

Classifications

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/676 Primary

  Bracketing for image capture at varying focusing conditions · CPC title

• H04N5/23212 Primary

  Electricity · mapped topic

• H04N5/2356

  Electricity · mapped topic

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title