Calibration of light-field camera geometry via robust fitting

Patent metadata

Field	Value
Publication number	US-9420276-B2
Application number	US-201514854493-A
Country	US
Kind code	B2
Filing date	Sep 15, 2015
Priority date	Feb 28, 2012
Publication date	Aug 16, 2016
Grant date	Aug 16, 2016

Abstract

Microlens positions for a light-field capture device may be calibrated. A calibration light-field image may be captured, with a microlens portion corresponding to each microlens of the light-field capture device. Interstitial spaces between the microlens portions may be identified and used to locate one or more center locations of the microlens portions. The center locations may be used to generate a model that indicates the microlens positions. Additionally or alternatively, the calibration light field image may be used to select one or more contour samples from among multiple contour samples of the microlens portions. The contour sample may be fitted to a circle centered at a center location of a microlens portion to identify the center location, which may then be used to generate a model that indicates the microlens positions. Multiple iterations may be used to enhance the accuracy of the models.

First claim

What is claimed is:

1. A method for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the method comprising:
in a data store, receiving a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
in a processor, identifying a plurality of interstitial spaces between the microlens portions to obtain a plurality of interstitial space locations; and
in the processor, using the interstitial space locations to generate a model that is indicative of the microlens positions, wherein the model can be applied in generating projected images from light-field data captured by the light-field image capture device.

2. The method of claim 1 , further comprising, in the processor, inverting and eroding the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein identifying the plurality of interstitial spaces comprises analyzing the inverted and eroded light-field image.

3. The method of claim 1 , wherein using the interstitial space locations to generate the model comprises using at least two of the interstitial space locations adjacent to a first microlens portion of the plurality of microlens portions to identify a first center location of a first center of the first microlens portion.

4. The method of claim 3 , wherein the model comprises a sheared Cartesian grid defined by parameters comprising at least one of the microlens positions, a slope between individual microlenses of the plurality of microlenses, and a spacing between individual microlenses of the plurality of microlenses.

5. The method of claim 3 , further comprising, after generation of the model:
in the processor, identifying a second center location of a second center of a second microlens portion of the plurality of microlens portions; and
in the processor, using the first center location and the second center location to modify the model to generate a modified model that is more accurately indicative of the microlens positions than the model.

6. The method of claim 5 , further comprising, after generation of the modified model:
in the processor, identifying a third center location of a third center of a third microlens portion of the plurality of microlens portions;
in the processor, identifying a fourth center location of a fourth center of a fourth microlens portion of the plurality of microlens portions; and
in the processor, using the third center location and the fourth center location to modify the modified model to generate a further modified model that is more accurately indicative of the microlens positions than the modified model.

7. The method of claim 6 , further comprising:
in the processor, eroding the calibration light-field image to generate an eroded light-field image;
in the processor, using the eroded light-field image to identify a fifth center location of a fifth center of a first microlens portion of the plurality of microlens portions;
in the processor, using the eroded light-field image to identify a sixth center location of a sixth center of a sixth microlens portion of the plurality of microlens portions;
in the processor, using the fifth center location and the sixth center location to generate a preliminary model that is indicative of the microlens positions; and
in the processor, inverting and eroding the calibration light-field image to generate an inverted and eroded light-field image in which the interstitial spaces have relatively high intensities;
wherein identifying the plurality of interstitial spaces comprises analyzing the inverted and eroded light-field image;
wherein each of identifying the first center of the first microlens portion, identifying the second center of the second microlens portion, identifying the third center of the third microlens portion, and identifying the fourth center of the fourth microlens portion comprises using the inverted and eroded light-field image.

8. A method for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, the method comprising:
in a data store, receiving a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
in a processor, using the calibration light-field image to select a first contour sample of a first microlens portion of the plurality of microlens portions from among a plurality of contour samples;
in the processor, fitting the first contour sample to a circle centered at a first center location of the first microlens portion to identify the first center location; and
in the processor, using the first center location to generate a model that is indicative of the microlens positions.

9. The method of claim 8 , further comprising, in the processor, prior to using the calibration light-field image to select the first contour sample of the first microlens portion, generating a preliminary model that is indicative of the microlens positions, wherein the model is more accurately indicative of the microlens positions than the preliminary model.

10. The method of claim 9 , wherein using the calibration light-field image to select the first contour sample comprises:
using the calibration light-field image to generate a resampled calibration light-field image with higher angular resolution than the light-field calibration image;
identifying the plurality of contour samples in the resampled calibration light-field image; and
selecting the first contour sample from within the resampled calibration light-field image.

11. The method of claim 10 , wherein using the calibration light-field image to select the first contour sample comprises applying the preliminary model to the resampled calibration light-field image to crop a window from the resampled calibration light-field image, wherein the window contains a majority of the first microlens portion.

12. The method of claim 8 , wherein selecting the first contour sample comprises:
determining that the first contour sample has a strong intensity gradient; and
determining that the first contour sample has an intensity oriented toward an optical center of the calibration light-field image.

13. The method of claim 8 , wherein selecting the first contour sample comprises rejecting outliers of the plurality of contour samples, wherein the outliers are not readily fitted to the circle.

14. The method of claim 13 , wherein rejecting the outliers comprises applying a RANSAC algorithm.

15. A non-transitory computer-readable medium for calibrating microlens positions for a light-field image capture device having a plurality of microlenses, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
causing a data store to receive a calibration light-field image captured by the light-field image capture device, the calibration light-field image comprising a plurality of microlens portions, each of which comprises microlens light-field data received through a single microlens of the plurality of microlenses;
identifying a plurality of interstitial spaces between the microlens portions to obtain a plurality of interstitial space locations; and

Assignees

• Lytro Inc

Inventors

• Liang Chia-Kai
• Wang Zejing

Classifications

• G02B27/0075

  with means for altering, e.g. increasing, the depth of field or depth of focus · CPC title

• H04N17/002 Primary

  for television cameras · CPC title

• G06T7/0018

  Physics · mapped topic

• H04N5/2254

  Electricity · mapped topic

• G02B3/0056

  arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses (G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085) · CPC title