Fixed pattern noise mitigation for a thermal imaging system

Patent metadata

Field	Value
Publication number	US-10230912-B2
Application number	US-201715629526-A
Country	US
Kind code	B2
Filing date	Jun 21, 2017
Priority date	Jun 28, 2016
Publication date	Mar 12, 2019
Grant date	Mar 12, 2019

Abstract

An imaging system whose Field of View FOV experiences occasional motion in relation to viewed scenes may be configured to reduce Fixed Pattern Noise (FPN) of acquired image data. FPN may be reduced by developing a pixel by pixel FPN correction term through a series of steps including blurring the image, identifying pixels to exclude from some calculations, a motion detector and an FPN updater for frames under motion and an FPN decay element for frames that are still.

First claim

What is claimed is:

1. A method for reducing Fixed Pattern Noise (FPN) in an imaging system including at least one imaging sensor and associated image signal processing chain, wherein successive frames of image pixel data are generated and passed to the image signal processing chain, and where the Field of View (FOV) of the imaging system is subject to occasional motion relative to imaged scenes, the method comprising:
selecting a group of pixels P x,y sig , the group of pixels P x,y sig comprising at least a portion of an image frame;
applying an FPN filter to generate an FPN correction term FPN x,y ; and
modifying individual pixels of P x,y sig based at least in part on FPN x,y from a previous image frame to generate a corrected set of pixels P x,y for calculating a value of FPN x,y for the current frame;
wherein applying the FPN filter comprises:
blurring at least some of the pixels P x,y to create a blurred pixel set K x,y and saving the blurred pixel set K x,y for at least one following frame, wherein K x,y values are available for the current frame and at least one previous frame during the application of the FPN filter;
identifying pixels that meet one or more predetermined criteria as excluded pixels for a current image frame;
detecting image motion by comparing K x,y values from the current frame and the at least one previous frame;
calculating, when motion is detected, an updated FPN x,y term based at least in part on P x,y and K x,y , for current non-excluded pixels and based at least in part on previous FPN x,y information for currently excluded pixels; and
when motion is not detected, decaying FPN x,y for both excluded and non-excluded pixels; and

wherein modifying individual pixels of P x,y sig based at least in part on FPN x,y from a previous image frame to generate a corrected set of pixels P x,y , comprises at least one of:
setting P x,y to P x,y sig −FPN x,y (t−1);
applying a clipping filter to at least a portion of the P x,y pixels; and
passing the P x,y pixels to other modules of the image signal processing chain.

2. The method of claim 1 , wherein the group of pixels P x,y sig comprises pixels at least one pixel away from the image frame boundaries.

3. The method of claim 1 , wherein blurring at least some of the pixels P x,y comprises:
applying a rolling kernel to the selected pixels; and
replacing a center pixel of each kernel with a pixel value derived from nearest neighbor pixels of the center pixel to create a blurred pixel K x,y .

4. The method of claim 3 , wherein creating the blurred pixel K x,y comprises calculating at least one of a mean, a median, a scaled mean, or a scaled median of the nearest neighbor pixels.

5. The method of claim 4 , wherein the kernel is a 3×3 kernel and K x,y is the median of the 8 pixels adjacent the center pixel in each kernel.

6. The method of claim 1 , wherein identifying pixels as excluded pixels comprises at least one of:
executing an amplitude filter on at least one of at least a portion of P x,y or K x,y pixels and excluding pixels above a predetermined amplitude; or
executing an edge filter on at least a portion of P x,y or K x,y pixels and excluding pixels whose edge filter results exceed a predetermined value.

7. The method of claim 6 , wherein the edge filter comprises at least one of a high-pass filter or an X-filter.

8. The method of claim 1 , wherein detecting motion comprises:
computing differences in at least a portion of the K x,y values from at least one previous frame to the next frame;
counting the number of pixels K x,y exceeding a predetermined difference threshold; and
determining there is motion if the number exceeds a predetermined count threshold.

9. The method of claim 1 , wherein a time to corresponds to the first frame for which FPN x,y is calculated, a time t corresponds to the current frame, and a time t−1 corresponds to the previous frame;
wherein calculating the updated FPN x,y term comprises:
setting FPN x,y (t 0 )=0;
setting FPN x,y (t) to at least one of FPN x,y (t−1)+S(P x,y (t)−K x,y (t)), or FPN x,y (t−1)−ave(FPN x,y (t−1))+S(P x,y (t)−K x,y (t)) for non-excluded pixels where motion between frames at t and t−1 is detected, where S is a predetermined scaling factor; and
setting FPN x,y (t)=FPN x,y (t−1) for excluded pixels where motion between frames at t and t−1 is detected; and

wherein decaying comprises setting FPN x,y (t)=D*FPN x,y (t−1) for all pixels where motion between frame at t and t−1 is not detected, where D is a predetermined decay factor.

10. An imaging system with Fixed Pattern Noise (FPN) reduction, the imaging system comprising;
at least one imaging sensor and associated image signal processing chain, wherein successive frames of image pixel data are generated and passed to the image signal processing chain, and where the Field of View (FOV) of the imaging system is subject to occasional motion relative to imaged scenes:
a selection element configured to select a group of pixels P x,y sig , the group of pixels P x,y sig comprising at least a portion of an image frame;
an FPN filter configured to generate an FPN correction term FPN x,y ; and
an FPN application element configured to modify individual pixels of P x,y sig based at least in part on FPN x,y from a previous image frame to generate a corrected set of pixels P x,y for calculating a value of FPN x,y for the current frame;
wherein the FPN filter includes:
a blurring element configured to blur at least some of the pixels P x,y to create a blurred pixel set K x,y and to save the blurred pixel set K x,y for at least one following frame, wherein K xy values are available for the current frame and at least one previous frame during application of the FPN filter;
an exclusion element configured to identify pixels that meet one or more predetermined criteria as excluded pixels for a current image frame;
a motion detection element configured to detect image motion by comparing K x,y values from the current frame and the at least one previous frame;
an FPN x,y update element configured to calculate, when motion is detected, an updated FPN x,y term based at least in part on P x,y and K x,y , for current non-excluded pixels and based at least in part on previous FPN x,y information for currently excluded pixels; and
an FPN x,y decay element configured to decay, when motion is not detected, FPN x,y for both excluded and non-excluded pixels; and

wherein the apply FPN x,y element is configured to generate the corrected set of pixels P x,y by at least one of
setting P x,y to P x,y sig −FPN x,y (t−1);
applying a clipping filter to at least a portion of the P x,y pixels; and
passing the P x,y pixels to the other modules of the image signal processing chain.

11. The system of claim 10 , wherein the group of pixels P x,y sig comprises pixels at least one pixel away from the image frame boundaries.

12. The system of claim 10 , wherein the blurring element is configured to:
apply a rolling kernel to the selected pixels; and
replace a center pixel of each kernel with a pixel value derived from nearest neighbor pixels of the center pixel to create a blurred pixel K x,y .

13. The system of claim 12 , wherein the blurring element is configured to create K x,y by calculating at least one of a mean, a median, a scaled mean, or a scaled median of the nearest neighbor pixels.

14. The system of claim 13 , wherein the kernel is a 3×3 kernel and K x,y is the median of the 8 pixels adjacent the center pixel in each kernel.

15. The system of claim 10 wherein the exclus

Assignees

• Seek Thermal Inc

Inventors

• Williams Ross
• Parrish William J
• Wolfe Jason

Classifications

• G06T7/13 Primary

  Edge detection · CPC title

• H04N5/33

  Transforming infrared radiation (cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20) · CPC title

• G06T5/002

  Physics · mapped topic

• G06T5/20

  using local operators · CPC title

• G06T2207/20192

  Edge enhancement; Edge preservation · CPC title