Multi-sensor alignment and real time distortion correction and image registration

Patent metadata

Field	Value
Publication number	US-10937193-B2
Application number	US-201816210918-A
Country	US
Kind code	B2
Filing date	Dec 5, 2018
Priority date	Dec 5, 2018
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

A method of multi-sensor alignment includes mechanically moving at least one sensor in a plurality of sensors to boresight and focus the plurality of sensors with one another so a respective optical center of each of the plurality of sensors is aligned and focused. The method includes collecting calibration images of a target wherein each sensor of the plurality of sensors collects at least one image of the target, and analytically comparing the images with one another to produce a look up table (LUT) of optical distortion and spatial registration for registration of image data from the plurality of sensors. The method includes saving the LUT in sensor memory for use in registering live images from the plurality of sensors.

First claim

What is claimed is:

1. A method of multi-sensor alignment comprising:
mechanically moving at least one sensor in a plurality of sensors to boresight and focus the plurality of sensors with one another so a respective optical center of each of the plurality of sensors is aligned and focused, wherein the plurality of sensors share a single entrance aperture;
collecting calibration images of a target wherein each sensor of the plurality of sensors collects at least one image of the target;
analytically comparing the images with one another to produce a look up table (LUT) of optical distortion and spatial registration for registration of image data from the plurality of sensors; and
saving the LUT in sensor memory for use in registering live images from the plurality of sensors.

2. The method as recited in claim 1 , wherein the plurality of sensors includes at least one sensor sensitive to short wave infrared (SWIR) wavelengths and at least one sensor sensitive to long wave infrared (LWIR) wavelengths.

3. The method as recited in claim 1 , wherein mechanically moving at least one sensor includes mechanically moving the at least one sensor in three orthogonal translational directions and three orthogonal rotational directions.

4. The method as recited in claim 1 , wherein the target includes a pattern formed with reflective paint and an emission pattern that are visible by the plurality of sensors.

5. The method as recited in claim 4 , wherein collecting calibration images of the target includes moving the target to different positions and/or distances in a field of view of the plurality of sensors and forming images with each sensor at each position and/or distance.

6. The method as recited in claim 1 , wherein analytically comparing includes correcting each image for distortion followed by scaling the images to a common scale.

7. The method as recited in claim 6 , wherein correcting each image for distortion includes correcting at least one of barrel shape distortion and/or pin cushion distortion.

8. The method as recited in claim 1 , wherein the LUT includes transformations for translating, including non-linear translation, of pixels in image data from at least one of the plurality of sensors to register with image data from another one of the plurality of sensors.

9. The method as recited in claim 1 , further comprising applying the LUT to register live multispectral images from the plurality of sensors.

10. A method of multi-spectral imaging comprising:
collecting image data from a plurality of sensors that are boresighted to one another with one or more beam splitters, wherein the plurality of sensors share a single entrance aperture;
applying a calibrated look up table (LUT) to the image data from at least one of the plurality of sensors to register the image data from all of the plurality of sensors together; and
outputting a composite image including registered image data from each sensor in the plurality of sensors.

11. The method as recited in claim 10 , wherein the plurality of sensors includes at least one sensor sensitive to short wave infrared (SWIR) wavelengths and at least one sensor sensitive to long wave infrared (LWIR) wavelengths.

12. The method as recited in claim 10 , wherein applying a calibrated LUT includes correcting at least one of barrel shape distortion and/or pin cushion distortion.

13. The method as recited in claim 10 , wherein the LUT includes transformations for translating, including non-linear translation, of pixels in image data from at least one of the plurality of sensors to register with image data from another one of the plurality of sensors.

14. The method as recited in claim 10 , wherein applying the calibrated LUT includes registering live multispectral images from the plurality of sensors.

15. An imaging system comprising:
a front end optic;
a plurality of sensors optically coupled to the front end optic by at least one beam splitter, wherein at least one of the sensors in the plurality of sensors is mounted to a mechanical jig configured to move the at least one sensor to boresight as well as focus respective optical centers of the plurality of sensors, wherein the plurality of sensors share a single entrance aperture; and
a controller and a memory each operatively connected to the plurality of imaging sensors to register image data from the plurality of sensors using a look up table (LUT) stored in the memory for registration of live multispectral images from the plurality of sensors.

16. The system as recited in claim 15 , wherein the plurality of sensors includes at least one sensor sensitive to short wave infrared (SWIR) wavelengths and at least one sensor sensitive to long wave infrared (LWIR) wavelengths.

17. The system as recited in claim 15 , wherein the mechanical jig includes mechanisms configured for moving at least one sensor in three orthogonal translational directions and three orthogonal rotational directions.

18. The system as recited in claim 15 , wherein the LUT includes transformations for correcting image data for distortion including correcting at least one of barrel shape distortion and/or pin cushion distortion.

19. The system as recited in claim 15 , wherein the LUT includes transformations for translating, including non-linear translation, of pixels in image data from at least one of the plurality of sensors to register with image data from another one of the plurality of sensors.

Assignees

• Goodrich Corp

Inventors

• Hu Haijun
• Eckhardt Andrew

Classifications

• H04N17/002 Primary

  for television cameras · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N23/58

  Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors · CPC title

• H04N23/20

  for generating image signals from infrared radiation only · CPC title