Super-resolution automatic target aimpoint recognition and tracking

Patent metadata

Field	Value
Publication number	US-11900562-B2
Application number	US-201916702279-A
Country	US
Kind code	B2
Filing date	Dec 3, 2019
Priority date	Nov 5, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

A system includes at least one imaging sensor configured to capture images of a target. The system also includes at least one controller configured to generate super-resolution images of the target using the captured images and identify multiple edges of the target using the super-resolution images. The at least one controller is also configured to identify an aimpoint on the target based on the identified edges of the target. In addition, the at least one controller is configured to update the aimpoint on the target as the target moves over time. The system may further include a high-energy laser (HEL) configured to generate an HEL beam that is directed towards the target, and the at least one controller may be configured to adjust one or more optical devices to direct the HEL beam at the identified aimpoint on the target.

First claim

What is claimed is:

1. A system comprising:
at least one imaging sensor configured to capture images of a moving target over a time period at a specified rate; and
at least one controller configured to:
upsample the captured images using interpolation of image data in the captured images to produce upsampled images of the moving target, the upsampled images having a higher resolution than the captured images, wherein the image data varies among the captured images due to movement of the moving target over the time period;
perform subpixel correlation using the upsampled images to produce aligned upsampled images of the moving target;
combine the aligned upsampled images to generate super-resolution images of the moving target;
identify multiple edges of the moving target using the super-resolution images;
identify an aimpoint on the moving target based on the identified edges of the moving target; and
update the aimpoint on the moving target as the moving target moves over time;

wherein, to identify the multiple edges of the moving target, the at least one controller is configured to use histogram binning and identify at least one median edge location in morphologically-filtered imagery produced using the super-resolution images; and
wherein, to combine the aligned upsampled images, the at least one controller is configured to:
combine multiple initial aligned upsampled images to produce a high-resolution reference image;
register additional aligned upsampled images to the high-resolution reference image; and
update the high-resolution reference image with the registered additional aligned upsampled images while applying recursive weighting factors to older aligned upsampled images in order to forget the older aligned upsampled images over time.

2. The system of claim 1 , wherein:
the system further comprises a target illumination laser (TIL) configured to generate a TIL beam that illuminates the moving target; and
the at least one imaging sensor is configured to capture images of the moving target containing reflected TIL energy.

3. The system of claim 1 , wherein, to identify the multiple edges of the moving target, the at least one controller is further configured to:
perform edge filtering in multiple directions in one or more of the super-resolution images to produce multiple edge-filtered images;
apply a threshold to the edge-filtered images to produce thresholded edge-filtered images;
merge the thresholded edge-filtered images to produce a binary edge detection map;
perform nonlinear morphological processing on the binary edge detection map to produce the morphologically-filtered imagery; and
identify the edges of the moving target using the at least one median edge location in the morphologically-filtered imagery.

4. The system of claim 1 , wherein the at least one controller is further configured to:
identify frame-to-frame shifts associated with the moving target in the captured images; and
adjust the identified aimpoint based on the frame-to-frame shifts.

5. The system of claim 1 , wherein:
the system further comprises a high-energy laser (HEL) configured to generate an HEL beam that is directed towards the moving target; and
the at least one controller is further configured to adjust one or more optical devices to direct the HEL beam at the identified aimpoint on the moving target.

6. At least one non-transitory computer readable medium containing instructions that when executed cause at least one processor to:
obtain captured images of a moving target over a time period at a specified rate;
upsample the captured images using interpolation of image data in the captured images to produce upsampled images of the moving target, the upsampled images having a higher resolution than the captured images, wherein the image data varies among the captured images due to movement of the moving target over the time period;
perform subpixel correlation using the upsampled images to produce aligned upsampled images of the moving target;
combine the aligned upsampled images to generate super-resolution images of the moving target;
identify multiple edges of the moving target using the super-resolution images;
identify an aimpoint on the moving target based on the identified edges of the moving target; and
update the aimpoint on the moving target as the moving target moves over time;
wherein the instructions that when executed cause the at least one processor to identify the multiple edges of the moving target comprise instructions that when executed cause the at least one processor to use histogram binning and identify at least one median edge location in morphologically-filtered imagery produced using the super-resolution images; and
wherein the instructions that when executed cause the at least one processor to combine the aligned upsampled images comprise instructions that when executed cause the at least one processor to:
combine multiple initial aligned upsampled images to produce a high-resolution reference image;
register additional aligned upsampled images to the high-resolution reference image; and
update the high-resolution reference image with the registered additional aligned upsampled images while applying recursive weighting factors to older aligned upsampled images in order to forget the older aligned upsampled images over time.

7. The at least one non-transitory computer readable medium of claim 6 , wherein the images of the moving target contain target illumination laser (TIL) energy reflected from the moving target.

8. The at least one non-transitory computer readable medium of claim 6 , wherein the instructions that cause the at least one processor to identify the multiple edges of the moving target further comprise:
instructions that when executed cause the at least one processor to:
perform edge filtering in multiple directions in one or more of the super-resolution images to produce multiple edge-filtered images;
apply a threshold to the edge-filtered images to produce thresholded edge-filtered images;
merge the thresholded edge-filtered images to produce a binary edge detection map;
perform nonlinear morphological processing on the binary edge detection map to produce the morphologically-filtered imagery; and
identify the edges of the moving target using the at least one median edge location in the morphologically-filtered imagery.

9. The at least one non-transitory computer readable medium of claim 6 , further containing instructions that when executed cause the at least one processor to:
identify frame-to-frame shifts associated with the moving target in the captured images; and
adjust the identified aimpoint based on the frame-to-frame shifts.

10. The at least one non-transitory computer readable medium of claim 6 , further containing instructions that when executed cause the at least one processor to:
adjust one or more optical devices to direct a high-energy laser (HEL) beam at the identified aimpoint on the moving target.

11. A method comprising:
obtaining captured images of a moving target over a time period at a specified rate;
upsampling the captured images using interpolation of image data in the captured images to produce upsampled images of the moving target, the upsampled images having a higher resolution than the captured images, wherein the image data varies among the captured images due to movement of the moving target over the time period;
performing subpixel correlation using the upsampled images to produce aligned upsampled images of the moving target;
combining the aligned upsampled images to generate super-resolution images of the moving target;
identifying multiple edges of the mo

Assignees

• Raytheon Co

Inventors

• Braunreiter Dennis C

Classifications

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06T3/4069

  by subpixel displacements · CPC title

• G06T3/4076

  using the original low-resolution images to iteratively correct the high-resolution images · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title