Image enhancement with fusion

Patent metadata

Field	Value
Publication number	US-10360711-B2
Application number	US-201715627165-A
Country	US
Kind code	B2
Filing date	Jun 19, 2017
Priority date	Dec 31, 2014
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

Various techniques are provided to combine visible and thermal image data. In one example, a method includes receiving visible image data and thermal image data for a scene. The method also includes extracting high spatial frequency content from the visible image data to provide filtered visible image data. The method also includes applying a corresponding gain to the filtered visible image data to provide weighted visible image data. The method also includes merging the weighted visible image data and the thermal image data to provide combined image data. Additional methods, systems, and other implementations are also provided.

First claim

The invention claimed is:

1. A method, comprising:
receiving visible image data and thermal image data for a scene, wherein the visible image data comprises a plurality of pixels;
determining a pixel spread value for each pixel, wherein the pixel spread values provide local smoothness metrics for the visible image data;
determining a corresponding gain for each pixel using the pixel spread value determined for the pixel;
extracting high spatial frequency content from the visible image data to provide filtered visible image data;
applying the corresponding gain determined for each pixel to corresponding pixels of the filtered visible image data to provide weighted visible image data; and
merging the weighted visible image data and the thermal image data to provide combined image data.

2. The method of claim 1 , wherein the determining the corresponding gain for each pixel comprises applying a function to the pixel spread values to provide the corresponding gain.

3. The method of claim 2 , wherein the function is a Gaussian function.

4. The method of claim 1 , wherein the extracting comprises applying a high pass spatial filter to the visible image data.

5. The method of claim 1 , further comprising low pass filtering the thermal image data prior to the merging.

6. The method of claim 1 , wherein the merging comprises superimposing the weighted visible image data on the thermal image data.

7. The method of claim 1 , wherein the merging comprises adding a luminance component of the weighted visible image data to the thermal image data.

8. The method of claim 1 , further comprising adding high resolution noise to the combined image data.

9. A system comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving visible image data and thermal image data for a scene, wherein the visible image data comprises a plurality of pixels;
determining a pixel spread value for each pixel, wherein the pixel spread values provide local smoothness metrics for the visible image data;
determining a corresponding gain for each pixel using the pixel spread value determined for the pixel;
extracting high spatial frequency content from the visible image data to provide filtered visible image data;
applying the corresponding gain determined for each pixel to corresponding pixels of the filtered visible image data to provide weighted visible image data; and
merging the weighted visible image data and the thermal image data to provide combined image data.

10. The system of claim 9 , wherein the determining the corresponding gain for each pixel comprises applying a function to the pixel spread values to provide the corresponding gain.

11. The system of claim 10 , wherein the function is a Gaussian function.

12. The system of claim 9 , wherein the extracting comprises applying a high pass spatial filter to the visible image data.

13. The system of claim 9 , wherein the operations further comprise applying a low pass filter to the thermal image data prior to the merging.

14. The system of claim 9 , wherein the merging comprises superimposing the weighted visible image data on the thermal image data.

15. The system of claim 9 , wherein the merging comprises adding a luminance component of the weighted visible image data to the thermal image data.

16. The system of claim 9 , wherein the operations further comprise adding high resolution noise to the combined image data.

Assignees

• Flir Systems

Inventors

• Lin Stephanie
• Hogasten Nicholas

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• G06K9/4604

  Physics · mapped topic

• G06T2207/10024

  Color image · CPC title