Infrared imaging enhancement with fusion

Patent metadata

Field	Value
Publication number	US-9635285-B2
Application number	US-201314138052-A
Country	US
Kind code	B2
Filing date	Dec 21, 2013
Priority date	Mar 2, 2009
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

Techniques using small form factor infrared imaging modules are disclosed. An imaging system may include visible spectrum imaging modules, infrared imaging modules, and other modules to interface with a user and/or a monitoring system. Visible spectrum imaging modules and infrared imaging modules may be positioned in proximity to a scene that will be monitored while visible spectrum-only images of the scene are either not available or less desirable than infrared images of the scene. Imaging modules may be configured to capture images of the scene at different times. Image analytics and processing may be used to generate combined images with infrared imaging features and increased detail and contrast. Triple fusion processing, including selectable aspects of non-uniformity correction processing, true color processing, and high contrast processing, may be performed on the captured images. Control signals based on the combined images may be presented to a user and/or a monitoring system.

First claim

What is claimed is:

1. A system comprising:
a memory adapted to receive a visible spectrum image of a scene from a visible spectrum imager and a plurality of infrared images of the scene from an infrared imager; and
a processor configured to communicate with the memory, wherein the processor is configured to:
generate a blurred infrared image from at least one of the plurality of infrared images;
determine, for each row of the blurred infrared image, a corresponding row fixed pattern noise (FPN) correction term;
determine, for each column of the blurred infrared image, a corresponding column FPN correction term;
apply the row and column FPN correction terms to the blurred infrared image to provide a row and column FPN corrected blurred infrared image;
determine a plurality of non-uniformity correction (NUC) terms based, at least in part, on the row and column FPN corrected blurred infrared image;
apply the NUC terms to one of the plurality of the infrared images to remove noise from the one of the plurality of the infrared images to provide a corrected infrared image;
receive control parameters;
derive high spatial frequency content from at least one of the visible spectrum image and the corrected infrared image; and
generate a combined image comprising relative contributions of at least the high spatial frequency content, wherein the relative contributions are determined in real-time by a user adjusting the control parameters.

2. The system of claim 1 , wherein the generating the blurred infrared image comprises:
detecting motion of the infrared imager based on motion sensor data provided by a motion sensor;
receiving a set of the plurality of infrared in response to detected motion;
generating a blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images.

3. The system of claim 1 , wherein:
the applying the NUC terms to the one of the plurality of infrared images comprises selectively applying the NUC terms to the one of the plurality of infrared images to selectively remove noise from the one of the plurality of infrared images; and
the selectively applying the NUC terms is determined by the control parameters.

4. The system of claim 1 , further comprising the infrared imager, wherein the infrared imager is configured to:
generate the blurred infrared image of the scene; and
derive the plurality of NUC terms.

5. The system of claim 1 , wherein:
one or more of the control parameters are determined by one or more threshold values associated with at least one of the visible spectrum image, the plurality of infrared images, and the corrected infrared image.

6. The system of claim 1 , further comprising a user interface configured to communicate with the memory and the processor, wherein the processor is configured to:
receive user input from the user interface; and
determine one or more of the control parameters from the user input.

7. The system of claim 1 , further comprising deriving color characteristics of the scene from at least one of the visible spectrum image and the infrared image, wherein:
the color characteristics of the scene are derived from a chrominance component of the visible spectrum image;
the relative contributions include a luminance component of the corrected infrared image; and
the relative contribution of the luminance component of the corrected infrared image substantially matches the relative contribution of the color characteristics, as controlled by the control parameters.

8. The system of claim 7 , wherein:
the luminance component of the corrected infrared image is blended with a luminance component of the visible spectrum image before being contributed to the combined image.

9. The system of claim 1 , wherein:
the high spatial frequency content is derived from a luminance component of the visible spectrum image;
the relative contributions include a chrominance component of the corrected infrared image; and
the relative contribution of the chrominance component of the corrected infrared image substantially matches the relative contribution of the high spatial frequency content, as controlled by the control parameters.

10. The system of claim 9 , wherein:
the high spatial frequency content is blended with a luminance component of the corrected infrared image before being contributed to the combined image.

11. The system of claim 1 , further comprising the infrared imager, wherein:
the infrared imager comprises a focal plane array (FPA) configured to capture the plurality of infrared images of the scene; and
the FPA comprises an array of microbolometers adapted to receive a bias voltage selected from a range of approximately 0.2 volts to approximately 0.7 volts.

12. A method comprising:
receiving a visible spectrum image of a scene from a visible spectrum imager and a plurality of infrared images of the scene from an infrared imager;
generating a blurred infrared image from at least one of the plurality of infrared images;
determining, for each row of the blurred infrared image, a corresponding row fixed pattern noise (FPN) correction term;
determining, for each column of the blurred infrared image, a corresponding column FPN correction term;
applying the row and column FPN correction terms to the blurred infrared image to provide a row and column FPN corrected blurred infrared image;
determining a plurality of non-uniformity correction (NUC) terms based, at least in part, on the row and column FPN corrected blurred infrared image;
applying the NUC terms to one of the plurality of the infrared images to remove noise from the one of the plurality of the infrared images to provide a corrected infrared image;
receiving control parameters;
deriving color characteristics of the scene from at least one of the visible spectrum image and the corrected infrared image;
and
generating a combined image comprising relative contributions of at least the color characteristics, wherein the relative contributions are determined in real-time by a user adjusting the control parameters.

13. The method of claim 12 , wherein the generating the blurred infrared image comprises:
detecting motion of the infrared imager based on motion sensor data provided by a motion sensor;
receiving a set of the plurality of infrared images in response to detected motion; and
generating the blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images.

14. The method of claim 12 , wherein the applying the NUC terms to the one of the plurality of infrared images comprises selectively applying the NUC terms to the one of the plurality of infrared images.

15. The method of claim 12 , wherein the infrared imager is configured to:
generate the blurred infrared image of the scene by intentionally blurring one of the plurality of infrared images of the scene or by determining a set of the plurality of infrared images corresponding to detected motion and generating the blurred infrared image from the set of the plurality of infrared images based on an average of the set of the plurality of infrared images; and
determine the plurality of NUC terms.

16. The method of claim 12 , wherein:
one or more of the control parameters are determined by one or more threshold value; associated with at least one of the visible spectrum image, the plurality of infrared images, and the corrected infrared image.

17. The method of claim 12 , further comprising:
receiving user input

Assignees

• Flir Systems

Inventors

• Teich Andrew C
• Högasten Nicholas
• Scott Jeffrey S
• Strandemar Katrin
• Nussmeier Mark
• Kurth Eric A
• Hoelter Theodore R
• Boulanger Pierre
• Sharp Barbara

Classifications

• G06T5/20 Primary

  using local operators · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T2207/20201

  Motion blur correction · CPC title

• G06T2207/10048

  Infrared image · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title