System and methods for calibration of an array camera

Patent metadata

Field	Value
Publication number	US-10958892-B2
Application number	US-202016785914-A
Country	US
Kind code	B2
Filing date	Feb 10, 2020
Priority date	Mar 10, 2013
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

Systems and methods for calibrating an array camera are disclosed. Systems and methods for calibrating an array camera in accordance with embodiments of this invention include the capturing of an image of a test pattern with the array camera such that each imaging component in the array camera captures an image of the test pattern. The image of the test pattern captured by a reference imaging component is then used to derive calibration information for the reference component. A corrected image of the test pattern for the reference component is then generated from the calibration information and the image of the test pattern captured by the reference imaging component. The corrected image is then used with the images captured by each of the associate imaging components associated with the reference component to generate calibration information for the associate imaging components.

First claim

What is claimed is:

1. A method for manufacturing an array camera device, the method comprising:
assembling an array of cameras comprising a plurality of imaging components that capture images of a scene from different viewpoints;
configuring the array of cameras to communicate with at least one processor;
configuring the processor to communicate with at least one type of memory; and
loading calibration information into the memory by:
loading a reference set of scene independent geometric corrections for image data captured by a first imaging component of the plurality of imaging components derived from test pattern image data captured by the first imaging component and data describing the test pattern using the processor; and
loading an associate set of scene independent geometric corrections for image data captured by a second imaging component of the plurality of imaging components, wherein the associate set of scene independent geometric corrections is derived using test pattern image data captured by the second imaging component and data for a corrected test pattern image, wherein the corrected test pattern image is captured by the first imaging component and corrected using the reference set of scene independent geometric corrections.

2. The method of claim 1 , wherein the first imaging component is a reference imaging component and the second imaging component is an associate imaging component, wherein the calibration information comprises:
reference calibration information for the reference imaging component comprising the scene independent geometric corrections for reference image data captured by the reference imaging component to account for distortions related to the mechanical construction of the reference imaging component and produce a corrected reference image; and
associate calibration information for the associate imaging component comprising the scene independent geometric corrections for associate image data captured by the associate imaging component that map locations of pixels in an image captured by the associate imaging component to corresponding pixel locations in the corrected reference image, where corresponding pixel locations represent the same point in a scene in the absence of disparity due to parallax.

3. The method of claim 2 , wherein the calibration information further comprises colorimetric corrections or photometric corrections for image data captured by one or more imaging components of the plurality of imaging components.

4. The method of claim 2 , further comprising loading a software application comprising machine readable instructions into the memory, where execution of the software application by the processor directs the processor to:
capture images of a scene using the plurality of imaging components in the array of cameras, wherein the captured images comprise:
an associate image captured by the associate imaging component; and
a reference image captured by the reference imaging component;

apply corrections to locations of pixels of the associate image using the associate calibration information;
generate a depth map by measuring disparity due to parallax between pixels in the reference image and corrected pixels in the associate image; and
synthesize an image using the generated depth map and at least some of the pixels from the captured images.

5. The method of claim 4 , wherein the execution of the software application by the processor further directs the processor to apply corrections to locations of pixels of the reference image using the reference calibration information; and
wherein the depth map is generated by measuring disparity due to parallax between the corrected pixels in the reference image and the corrected pixels in the associate image.

6. The method of claim 1 further comprising capturing the test pattern image data using the array of cameras, wherein the test pattern is placed at a defined distance away from the array of cameras when the images of the test pattern are captured, and the distance is at least 70 percent of a hyperfocal distance of the array of cameras.

7. The method of claim 1 further comprising capturing the test pattern image data using the array of cameras, wherein the test pattern is placed at a defined distance away from the array of cameras when the images of the test pattern are captured, and the distance is at least 50 percent of a hyperfocal distance of the array of cameras.

8. The method of claim 1 , wherein the test pattern includes a low-contrast slanted edge pattern.

9. The method of claim 8 , wherein the test pattern includes a plurality of Macbeth Color Chart type patterns inset at different positions in the low-contrast slanted pattern.

10. The method of claim 1 , further comprising performing at least one pass/fail test of the array of cameras based on captured images of the test pattern to verify proper image capture by the plurality of imaging components.

11. The method of claim 1 further comprising generating the reference set of scene independent geometric corrections by:
identifying reference intersection points in the image of the test pattern captured by the reference imaging component;
determining uniformity characteristics of the reference imaging component from reference intersection points and the test pattern; and
deriving parameters for the reference imaging component to compensate for low frequency aberrations in the image of the test pattern captured by the reference imaging component.

12. The method of claim 1 , wherein the first imaging component is a reference imaging component and the second imaging component is an associate imaging component, wherein the method further comprises generating the associate set of scene independent geometric corrections by:
identifying associate intersection points in images of the test pattern captured by the associate imaging component;
translating associate intersection points in accordance with an expected parallax shift for the associate imaging component relative to the reference imaging component; and
deriving parameters for the associate imaging component to compensate for low frequency aberrations in the image of the test pattern captured by the associate imaging component by comparing translated associate intersection points to corresponding intersection points in the corrected test pattern image for the reference imaging component.

13. The method of claim 12 , wherein the expected parallax shift for the associate imaging component is based upon at least one of the physical offset of the associate imaging component to the reference imaging component, the behavior of sensor optics in the associate imaging component, and a distance of the test pattern from the array of cameras.

14. The method of claim 1 , further comprising:
generating colorimetric corrections or photometric corrections for image data captured by each imaging component in the array of cameras using test pattern image data captured by each imaging component using the processor; and
storing the generated colorimetric corrections or photometric corrections in the memory.

15. The method of claim 1 , wherein the array of cameras includes more than one reference imaging component.

16. The method of claim 1 , wherein the processor includes a graphics processing unit.

17. The method of claim 1 , wherein the associate set of scene independent geometric corrections are represented by a grid that provides a geometric correction prescription for pixels of the second imaging component.

18. The method of claim 1 , wherein

Assignees

• Fotonation Ltd

Inventors

• Mullis Robert

Classifications

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N13/246 Primary

  Calibration of cameras · CPC title

• H04N13/282

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title