Extended color processing on pelican array cameras

Patent metadata

Field	Value
Publication number	US-9497429-B2
Application number	US-201314145734-A
Country	US
Kind code	B2
Filing date	Dec 31, 2013
Priority date	Mar 15, 2013
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

Systems and methods for extended color processing on Pelican array cameras in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating a high resolution image includes obtaining input images, where a first set of images includes information in a first band of visible wavelengths and a second set of images includes information in a second band of visible wavelengths and non-visible wavelengths, determining an initial estimate by combining the first set of images into a first fused image, combining the second set of images into a second fused image, spatially registering the fused images, denoising the fused images using bilateral filters, normalizing the second fused image in the photometric reference space of the first fused image, combining the fused images, determining a high resolution image that when mapped through a forward imaging transformation matches the input images within at least one predetermined criterion.

First claim

What is claimed is:

1. A method of generating a high resolution image of a scene using an imager array including a plurality of imagers that each capture an image of the scene, and a forward imaging transformation for each imager, the method comprising:
obtaining input images captured by a plurality of imagers, where a first set of input images includes image information captured in a first band of visible wavelengths and a second set of input images includes image information captured in a second band of visible wavelengths and non-visible wavelengths;
determining an initial estimate of at least a portion of a high resolution image using a processor configured by software to:
combine image information from the first set of input images into a first fused image;
combine image information from the second set of input images into a second fused image, wherein the first fused image and the second fused image have the same resolution and the resolution is higher than the resolution of any of the input images;
spatially register the first fused image and the second fused image;
denoise the first fused image using a first bilateral filter;
denoise the second fused image using a second bilateral filter;
normalize the second fused image in the photometric reference space of the first fused image; and
combine the first fused image and the second fused image into an initial estimate of at least a portion of the high resolution image; and

determining a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image using the processor configured by software;
wherein each forward imaging transformation corresponds to the manner in which each imager in the imaging array generated the input images; and
wherein the high resolution image has a resolution that is greater than any of the input images.

2. The method of claim 1 , wherein the first band of visible wavelengths and the second band of visible and non-visible wavelengths have some degree of overlap.

3. The method of claim 1 , wherein the second band of visible and non-visible wavelengths includes green, red, and near-infrared light.

4. The method of claim 1 , wherein:
the first set of input images are captured by a first set of imagers from the plurality of imagers and the first set of imagers are sensitive to light in the first band of visible wavelengths; and
the second set of input images are captured by a second set of imagers from the plurality of imagers and the second set of imagers are sensitive to light in the second band of visible and non-visible wavelengths.

5. The method of claim 4 , wherein the processor being configured to combine image information from the first set of input images into a first fused image utilizes analog gain and noise information from the first set of imagers and the processor being configured to combine image information from the second set of input images into a second fused image utilizes analog gain and noise information from the second set of imagers.

6. The method of claim 1 , wherein the first bilateral filter and the second bilateral filter utilize weights that are a function of both the photometric and geometric distance between a pixel and pixels in the neighborhood of the pixel.

7. The method of claim 1 , wherein the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second fused image.

8. The method of claim 1 , wherein the first set of input images are captured by a first set of imagers from the plurality of imagers and the first bilateral filter is a cross-channel bilateral filter utilizing weights determined for the second fused image when an analog gain value of the first set of imagers is above a predetermined threshold.

9. The method of claim 1 , wherein normalizing the second fused image in the photometric reference space of the first fused image comprises applying gains and offsets to pixels of the second fused image.

10. The method of claim 9 , wherein the gain for each pixel of the second fused image is determined by the equation:

a
^

=

⌊

∑
r

⁢

⁢

∑
c

⁢

⁢

g
⁡

(

r
,
c

)

·

e
⁡

(

r
,
c

)

⌋

-

N
r

⁢

N
c

⁢

g
_

·

e
_

[

∑
r

⁢

⁢

∑
c

⁢

e
2

Assignees

• Pelican Imaging Corp

Inventors

• Mullis Robert
• Lelescu Dan
• Venkataraman Kartik

Classifications

• H04N5/265 Primary

  Mixing · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N23/13

  with multiple sensors · CPC title

• H04N23/15 Primary

  Image signal generation with circuitry for avoiding or correcting image misregistration · CPC title

• H04N9/093 Primary

  Electricity · mapped topic