Robust autofocus algorithm for multi-spectral imaging systems

Patent metadata

Field	Value
Publication number	US-9354489-B2
Application number	US-201414177175-A
Country	US
Kind code	B2
Filing date	Feb 10, 2014
Priority date	Feb 10, 2014
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

An autofocus metric approach for focusing video images, automatically, based on images taken during a focus sweep in which a focus cell is repositioned for each of the images is provided. The approach includes, given an edge detected image from the focus sweep and an associated focus cell position in the focus sweep, an autofocus engine dividing the edge detected image into sub-images. For each sub-image, the autofocus engine calculates a normalized edge detection strength and compares it to a threshold. Based on the comparison, the autofocus engine determines whether an edge is present in the sub-image. Based on the determinations of edges in the sub-images, the autofocus engine calculates an autofocus metric associated with the given focus cell position. The autofocus engine provides the autofocus metric together with autofocus metrics associated with other focus cell positions to focus the video images.

First claim

What is claimed is:

1. A method for focusing video images, automatically, based on images taken during a focus sweep in which a focus cell is repositioned for each of the images, the method comprising:
in an autofocus engine, given an edge detected image from a focus sweep and an associated position of a focus cell in the focus sweep, dividing the edge detected image into sub-images;
for each sub-image, calculating a normalized edge detection strength of a subject sub-image, wherein calculating the normalized edge detection strength comprises
selecting a maximum pixel value found in the subject sub-image and a minimum pixel value found in the subject sub-image,
forming a ratio of a difference of the maximum pixel value and minimum pixel value to a sum of the maximum pixel value and minimum pixel value, the ratio being a contrast measure, and
multiplying the contrast measure with a mean of the pixel values in the sub-image resulting in a normalized and weighted edge detection strength of the subject sub-image;

comparing the normalized edge detection strength of the subject sub-image to a threshold;
determining, based on the comparison, an edge is present in the subject sub-image;
calculating, based on the determinations of edges in the sub-images, an autofocus metric associated with the given position of the focus cell; and
providing the autofocus metric together with autofocus metrics associated with other positions of the focus cell to focus video images.

2. The method of claim 1 wherein calculating the autofocus metric includes averaging the normalized edge detection strengths of sub-images having edges present.

3. The method of claim 1 further comprising:
selecting a position of the focus cell based on the autofocus metrics; and
repositioning the focus cell to the selected position to focus the video images.

4. The method of claim 3 wherein selecting the position of the focus cell includes:
creating a model of autofocus metrics and positions of the focus cell from the calculated autofocus metrics and associated positions of the focus cell; and
selecting a best position of the focus cell from the model.

5. The method of claim 4 wherein the model is a non-linear regression fit of the calculated autofocus metrics to the model.

6. The method of claim 1 further comprising capturing images, during a focus sweep, in wavelengths selected from a group image sensors consisting of Day Television (DTV), Near Infrared (NIR), SWIR-Short Wave Infrared (SWIR), Mid Wave Infrared (MWIR), and Long Wave Infrared (LWIR).

7. A method for focusing video images, automatically, based on images taken during a focus sweep in which a focus cell is repositioned for each of the images, the method of claim 1 further comprising:
in an autofocus engine, given an edge detected image from a focus sweep and an associated position of a focus cell in the focus sweep, dividing the edge detected image into sub-images;
for each sub-image, calculating a normalized edge detection strength of a subject sub-image;
comparing the normalized edge detection strength of the subject sub-image to a threshold;
determining, based on the comparison, an edge is present in the subject sub-image;
calculating, based on the determinations of edges in the sub-images, an autofocus metric associated with the given position of the focus cell;
providing the autofocus metric together with autofocus metrics associated with other positions of the focus cell to focus video images;
for each image taken during a focus sweep, cropping a subject image into a region of interest;
reducing white noise of the region of interest resulting in a filtered image; and
detecting edges in the filtered image resulting in an edge detected image associated with a position of the focus cell in the focus sweep.

8. The method of claim 7 wherein reducing white noise includes applying a bilateral or median filter to pixels of the region of interest.

9. The method of claim 7 wherein detecting edges includes applying Sobel or Laplace edge detection.

10. The method of claim 7 , wherein calculating the autofocus metric includes averaging the normalized edge detection strengths of sub-images having edges present.

11. The method of claim 7 , further comprising:
selecting a position of the focus cell based on the autofocus metrics; and
repositioning the focus cell to the selected position to focus the video images.

12. The method of claim 11 , wherein selecting the position of the focus cell includes:
creating a model of autofocus metrics and positions of the focus cell from the calculated autofocus metrics and associated positions of the focus cell; and
selecting a best position of the focus cell from the model.

13. The method of claim 12 , wherein the model is a non-linear regression fit of the calculated autofocus metrics to the model.

14. The method of claim 7 , further comprising capturing images, during a focus sweep, in wavelengths selected from a group image sensors consisting of Day Television (DTV), Near Infrared (NIR), SWIR-Short Wave Infrared (SWIR), Mid Wave Infrared (MWIR), and Long Wave Infrared (LWIR).

15. A system for focusing video images, automatically, based on images taken during a focus sweep in which a focus cell is repositioned for each of the images, the system comprising:
a memory having computer executable instructions thereupon;
at least one interface receiving an edge detected image from a focus sweep and an associated position of a focus cell in the focus sweep;
an autofocus engine coupled to the memory and the at least one interface, the computer executable instructions when executed by the autofocus engine cause the autofocus engine to:
divide the edge detected image into sub-images;
for each sub-image, calculate a normalized edge detection strength of a subject sub-image, wherein the autofocus engine is configured to calculate the normalized edge detection strength by
selecting a maximum pixel value found in the subject sub-image and a minimum pixel value found in the subject sub-image,
forming a ratio of a difference of the maximum pixel value and minimum pixel value to a sum of the maximum pixel value and minimum pixel value, the ratio being a contrast measure, and
multiplying the contrast measure with a mean of the pixel values in the sub-image resulting in a normalized and weighted edge detection strength of the subject sub-image;

compare the normalized edge detection strength of the subject sub-image to a threshold;
determine, based on the comparison, an edge is present in the subject sub-image;
calculate, based on the determinations of edges in the sub-images, an autofocus metric associated with the given position of the focus cell; and
provide the autofocus metric together with autofocus metrics associated with other positions of the focus cell to focus video images.

16. The system of claim 15 , wherein the computer executable instructions when executed by the autofocus engine additionally cause the autofocus engine to:
for each image taken during a focus sweep, crop a subject image into a region of interest;
reduce white noise of the region of interest resulting in a filtered image; and
detect edges in the filtered image resulting in an edge detected image associated with a position of the focus cell in the focus sweep.

17. A tangible non-transitory computer-readable storage medium having computer readable instructions stored therein for focusing video images, automatically, based on images taken during a focus sweep in which a focus cell is repositioned for each

Assignees

• Raytheon Co

Inventors

• Fluckiger David U

Classifications

• G03B13/36 Primary

  Autofocus systems · CPC title

• H04N23/673 Primary

  based on contrast or high frequency components of image signals, e.g. hill climbing method · CPC title

• H04N5/23212

  Electricity · mapped topic