Automated change assessment of target objects during a dynamic ISR mission

Patent metadata

Field	Value
Publication number	US-12266169-B2
Application number	US-202217810948-A
Country	US
Kind code	B2
Filing date	Jul 6, 2022
Priority date	Jul 6, 2022
Publication date	Apr 1, 2025
Grant date	Apr 1, 2025

Abstract

Automated real-time aerial change assessment of targets is provided. An aerial image of a target area is recorded during a flyover and a target detected in the aerial image. A sequence of images of the target area are recorded during a subsequent flyover. The system determines a target detection probability according to confidence scores of the sequence of images and determines a change status of the target. Responsive to a target change, a percentage of change is determined according to image feature matching between first aerial image and each of the images from the second flyover. Target detection probability and percentage of change are combined as statistically independent events to determine a probability of change. The probability of change and percentage of change for each image in the sequence is output in real-time, and final change assessment is output when the aircraft exits the target area.

First claim

What is claimed is:

1. A computer-implemented method for automated change assessment of target objects, the method comprising using a number of processors to perform operations of:
recording an aerial image of a target area during a first flyover;
detecting a target object in the aerial image with a corresponding confidence score above a detection threshold;
recording a sequence of aerial images of the target area during a subsequent second flyover;
calculating a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;
applying a penalty score to the transition probability in favor of object detection;
updating a percentage of change and the probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;
determining in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;
determining in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;
responsive to a determination that the target object has changed, determining by feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;
combining the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and
outputting in real-time the probability of change and percentage of change for each image in the sequence of images.

2. The method of claim 1 , further comprising recording a geographical position of the target object.

3. The method of claim 2 , further comprising outputting a final change assessment according to the percentage of change of the target object when a flight camera field of view exits a geographical boundary of the target area.

4. The method of claim 1 , wherein determining the percentage change of the target object further comprises hidden Markov chain normalization of the percentage of change of the target object.

5. The method of claim 1 , wherein the method is performed onboard an unmanned aerial vehicle.

6. The method of claim 1 , further comprising determining a geographical boundary of the target area based on a geographical position of the target object and a flight camera field of view, wherein the geographical boundary of the target area is used during the second flyover to ensure the geographical position of the target object stays within corner pixels of each of the sequence of aerial images.

7. The method of claim 1 , wherein the feature matching comprises scale and rotation invariant key point matching.

8. A computer-implemented method for real-time automated change assessment by a flight computer onboard an unmanned aerial vehicle, the method comprising performing, using a number of processors onboard the unmanned aerial vehicle, operations of:
recording, with an onboard flight camera, an aerial image of a target area during a first flyover;
detecting a target object in the aerial image with a corresponding confidence score above a detection threshold;
recording, with the onboard flight camera, a sequence of aerial images of the target area during a subsequent second flyover;
calculating a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;
applying a penalty score to the transition probability in favor of object detection;
updating a percentage of change and probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;
determining in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;
determining in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;
responsive to a determination that the target object has changed, determining by image feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;
combining the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and
outputting in real-time the probability of change and percentage of change for each image in the sequence of images.

9. The method of claim 8 , further comprising recording a geographical position of the target object.

10. The method of claim 9 , further comprising outputting a final change assessment according to the percentage of change of the target object when the unmanned aerial vehicle exits a geographical boundary of the target area.

11. The method of claim 8 , wherein determining the percentage change of the target object further comprises hidden Markov chain normalization of the percentage of change of the target object.

12. The method of claim 8 , further comprising determining a geographical boundary of the target area based on a geographical position of the target object and a field of view of the onboard flight camera, wherein the geographical boundary of the target area is used during the second flyover to ensure the geographical position of the target object stays within corner pixels of each of the sequence of aerial images.

13. A system configured to assess change of target objects, wherein the system comprises:
a storage device configured to store program instructions; and
one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to:
record an aerial image of a target area during a first flyover;
detect a target object in the aerial image with a corresponding confidence score above a detection threshold;
record a sequence of aerial images of the target area during a subsequent second flyover;
calculate a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;
apply a penalty score to the transition probability in favor of object detection;
update a percentage of change and probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;
determine in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;
determine in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;
responsive to a determination that the target object has changed, determine based upon image feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;
combine the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and
output in real-time the probability of change and percentage of change for each image in the sequence of images.

14. The system of claim 13 , wherein the

Assignees

• Boeing Co

Inventors

• Yang Yan
• Foster Paul

Classifications

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06V2201/07

  Target detection · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• G06V10/757

  Matching configurations of points or features · CPC title