Calibration of plenoptic imaging systems using fourier transform

Patent metadata

Field	Value
Publication number	US-9613417-B2
Application number	US-201514638907-A
Country	US
Kind code	B2
Filing date	Mar 4, 2015
Priority date	Mar 4, 2015
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

Calibration for a plenoptic imaging system. The plenoptic imaging system includes a detector array that is subdivided into superpixels. A plenoptic image captured by the detector array of the plenoptic imaging system is accessed. For a row of superpixels, a slice is selected through the row, the selected slice having a Fourier transform with a stronger fundamental component compared to other slices through the row. A pitch of the row of superpixels is determined based on a frequency of the fundamental component of the selected slice. A rotation of the row of superpixels is determined based on a rotation of the selected slice.

First claim

What is claimed is:

1. A method for calibrating a plenoptic imaging system, the plenoptic imaging system including a microlens array and a detector array, the plenoptic imaging system subdividing the detector array into superpixels, the method comprising a processor executing the steps of:
accessing a plenoptic image captured by the detector array of the plenoptic imaging system;
for a row of superpixels in the plenoptic image, selecting a one-dimensional slice through the row, the selected slice having a one-dimensional Fourier transform with a stronger fundamental component compared to other one-dimensional slices through the row; and
determining a pitch of the row of superpixels based on a frequency of the fundamental component of the selected slice, and determining a rotation of the row of superpixels based on a rotation of the plenoptic image relative to the selected slice, wherein the determined pitch is a pitch of the microlens array in terms of pixel size on the detector array and the determined rotation is a rotation of the microlens array relative to the detector array.

2. The method of claim 1 , the method further comprising:
determining a plurality of different rotation angles between the selected slice and the plenoptic image;
determining a spectral power of the slice at each of the plurality of rotation angles; and
selecting the rotation angle associated with the highest spectral power, the selected rotation angle being the rotation of the row of superpixels.

3. The method of claim 2 , wherein determining a spectral power of the slice at each of the plurality of rotation angles further comprises adjusting each of the spectral powers for spectral leakage.

4. The method of claim 3 , wherein adjusting a spectral power for spectral leakage comprises:
for a particular rotation angle,
determining a spectral power of the slice for different sample sizes of the slice; and
selecting the highest spectral power, of the determined spectral powers.

5. The method of claim 1 , wherein determining a pitch of the row of superpixels based on a frequency of the fundamental component of the selected slice is done for a plurality of different focus settings.

6. The method of claim 1 , wherein the slice spans the entire row of superpixels.

7. The method of claim 1 , further comprising:
sampling the plenoptic image using a plurality of one-dimensional slices, including the selected slice, where each slice is through a different row of superpixels;
determining the spectral power for each of the slices, and
the selected slice is the slice having the highest of the determined spectral powers.

8. A plenoptic imaging system comprising:
an image-forming optical module characterized by a pupil plane and an image plane;
an array of micro-imaging elements located at the image plane or a conjugate thereof;
a detector array located at a conjugate of the pupil plane, wherein the detector array is subdivided into superpixels;
a processor configured to execute modules;
a memory storing the modules, the modules comprising a processing module, the processing module configured to:
access a plenoptic image captured by the detector array of the plenoptic imaging system;
for a row of superpixels in the plenoptic image, select a one-dimensional slice through the row, the selected slice having a one-dimensional Fourier transform with a stronger fundamental component compared to other one-dimensional slices through the row; and
determine a pitch of the row of superpixels based on a frequency of the fundamental component of the selected slice, and determine a rotation of the row of superpixels based on a rotation of the plenoptic image relative to the selected slice, wherein the determined pitch is a pitch of the array of micro-imaging elements in terms of pixel size on the detector array and the determined rotation is a rotation of the array of micro-imaging elements relative to the detector array.

9. The system of claim 8 , wherein the processing module is further configured to:
determine a plurality of different rotation angles between the selected slice and the plenoptic image;
determine a spectral power of the slice at each of the plurality of rotation angles; and
select the rotation angle associated with the highest spectral power, the selected rotation angle being the rotation of the row of superpixels.

10. The system of claim 9 , wherein the processing module is further configured to:
adjust each of the determined spectral powers for spectral leakage.

11. The system of claim 10 , wherein the processing module is further configured to:
for a particular rotation angle,
determine a spectral power of the slice for different sample sizes of the slice; and
select the highest spectral power, of the determined spectral powers.

12. The system of claim 8 , wherein the processing module is further configured to:
determine a pitch of the row of superpixels based on a frequency of the fundamental component of the selected slice for a plurality of different focus settings.

13. The system of claim 8 , wherein the slice spans the entire row of superpixels.

14. The system of claim 8 , wherein the processing module is further configured to:
sample the plenoptic image using a plurality of one-dimensional slices, including the selected slice, where each slice is through a different row of superpixels;
determine the spectral power for each of the slices, and
the selected slice is the slice having the highest of the determined spectral powers.

15. A non-transitory computer-readable storage medium storing executable computer program instructions for calibrating a plenoptic imaging system that includes a microlens array and a detector array, the plenoptic imaging system subdividing the detector array into superpixels, the instructions executable by the plenoptic imaging system to perform steps comprising:
accessing a plenoptic image captured by the detector array of the plenoptic imaging system;
for a row of superpixels in the plenoptic image, selecting a one-dimensional slice through the row, the selected slice having a one-dimensional Fourier transform with a stronger fundamental component compared to other one-dimensional slices through the row; and
determining a pitch of the row of superpixels based on a frequency of the fundamental component of the selected slice, and determining a rotation of the row of superpixels based on a rotation of the plenoptic image relative to the selected slice, wherein the determined pitch is a pitch of the microlens array in terms of pixel size on the detector array and the determined rotation is a rotation of the microlens array relative to the detector array.

16. The computer-readable storage medium of claim 15 , further comprising:
determining a plurality of different rotation angles between the selected slice and the plenoptic image;
determining a spectral power of the slice at each of the plurality of rotation angles; and
selecting the rotation angle associated with the highest spectral power, the selected rotation angle being the rotation of the row of superpixels.

17. The computer-readable storage medium of claim 16 , wherein determining a spectral power of the slice at each of the plurality of rotation angles further comprises adjusting each of the spectral powers for spectral leakage.

18. The computer-readable storage medium of claim 17 , wherein adjusting a spectral power for spectral leakage comprises:
for a particular rotation angle,
determining a spectral power of the slice for different sample siz

Assignees

• Namboodiri Vipin
• Prasad Agara Venkatesha Rao Krishna
• Ricoh Co Ltd

Inventors

• Namboodiri Vipin
• Prasad Agara Venkatesha Rao Krishna

Classifications

• G06T2207/20056

  Discrete and fast Fourier transform, [DFT, FFT] · CPC title

• H04N13/296

  Synchronisation thereof; Control thereof · CPC title

• G03B41/00

  Special techniques not covered by groups G03B31/00 - G03B39/00; Apparatus therefor · CPC title

• G06T7/85

  Stereo camera calibration · CPC title

• G06T2207/10052

  Images from lightfield camera · CPC title