Methods and system for infrared tracking

Patent metadata

Field	Value
Publication number	US-11748898-B2
Application number	US-202218064771-A
Country	US
Kind code	B2
Filing date	Dec 12, 2022
Priority date	Nov 3, 2017
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A method for tracking includes obtaining an infrared image and a visible image from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), combining the infrared image and the visible image to obtain a combined image, identifying a target in the combined image, and controlling at least one of the UAV, the carrier, or the imaging device to track the identified target. Combing the infrared image and the visible image includes matching the infrared image and the visible image based on matching results of different matching methods.

First claim

What is claimed is:

1. A method for tracking, comprising:
obtaining an infrared image and a visible image from an imaging device supported by a carrier of an unamend aerial vehicle (UAV);
combining the infrared image and the visible image to obtain a combined image, including:
matching the infrared image and the visible image based on matching results of different matching methods, including:
obtaining a first matching result by a first matching method;
obtaining a second matching result by a second matching method;
combining, linearly or non-linearly, the first matching result having a first weight and the second matching result having a second weight to derive a final matching result; and
matching the infrared image and the visible image based on the final matching result;

identifying a target in the combined image; and
controlling at least one of the UAV, the carrier, or the imaging device to track the identified target.

2. The method of claim 1 , wherein matching the infrared image and the visible image includes:
processing the infrared image to extract infrared features;
processing the visible image to extract the visible features; and
matching the infrared image and the visible image based on the infrared features and the visible features.

3. The method of claim 2 , wherein combining the infrared image and the visible image includes:
applying at least one of the visible features to the infrared image to generate the combined image based on the matching.

4. The method of claim 3 , wherein applying the at least one of the visible features to the infrared image includes:
determining the at least one of the visible features based on prominence of the visible features, wherein prominence of a feature is positively correlated to at least one of a size of the feature or obviousness of an edge of the feature.

5. The method of claim 4 , wherein determining the at least one of the visible features based on the prominence of the visible features includes:
sorting the visible features of the visible image in a rank of prominence from high to low or low to high; and
determining one or more visible features of highest prominence in the rank as the at least one visible features to be applied to the infrared image.

6. The method of claim 3 , wherein applying the at least one of the visible features to the infrared image includes:
determining an intensity of the at least one of the visible features; and
applying the at least one of the visible features having the determined intensity to the infrared image.

7. The method of claim 2 , wherein:
the infrared features include an infrared outer contour and the visible features include a visible outer contour; and
matching the infrared image and the visible image further includes aligning the infrared outer contour with the visible outer contour.

8. The method of claim 1 , wherein identifying the target in the combined image includes:
obtaining target information corresponding to the target; and
identifying the target based on the target information.

9. The method of claim 8 , wherein obtaining target information corresponding to the target includes:
obtaining the target information based on a user operation through a remote terminal.

10. The method of claim 9 , wherein the target information includes at least one of a size, a color, or a texture of the target.

11. The method of claim 1 , wherein identifying the target in the combined image includes:
extracting one or more target features of the target in a pre-obtained image; and
identifying the target in the combined image based on the one or more target features.

12. The method of claim 1 , wherein controlling the at least one of the UAV, the carrier, or the imaging device to track the identified target includes detecting a deviation of the target from a predetermined configuration in the combined image and generating the control signals based on the deviation.

13. The method of claim 12 , wherein the predetermined configuration includes at least one of a predetermined position or a predetermined size.

14. The method of claim 1 , further comprising:
displaying a graphical tracking indicator corresponding to the target in the combined image.

15. The method of claim 1 , further comprising:
determining an adjusted operation parameter of the at least one of the UAV, the carrier, or the imaging device to track the identified target;
determining whether the adjusted operation parameter compliance operation limits of the UAV, the carrier, and the imaging device; and
modifying the adjusted operation parameter in response to determining that the adjusted operation parameter does not compliance the operation limits.

16. The method of claim 15 , further comprising:
generating and outputting a warning indicator in response to modifying the adjusted operation parameter, the waring indicator including at least one of a textual, audio, visual, or mechanical signal.

17. An unmanned aerial vehicle (UAV), comprising:
a carrier;
a memory that stores one or more computer-executable instructions; and
one or more processors configured to access the memory and execute the computer-executable instructions to:
obtain, from an imaging device supported by the carrier, an infrared image and a visible image;
combine the infrared image and the visible image to obtain a combined image, including:
matching the infrared image and the visible image based on matching results of different matching methods, including:
obtaining a first matching result by a first matching method;
obtaining a second matching result by a second matching method;
combining, linearly or non-linearly, the first matching result having a first weight and the second matching result having a second weight to derive a final matching result; and
matching the infrared image and the visible image based on the final matching result;

identify a target in the combined image; and
control at least one of the UAV, the carrier, or the imaging device to track the identified target.

18. A tracking system, comprising:
a memory that stores one or more computer-executable instructions; and
one or more processors configured to access the memory and execute the computer-executable instructions to:
obtain, from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), an infrared image and a visible image;
combine the infrared image and the visible image to obtain a combined image, including:
matching the infrared image and the visible image based on matching results of different matching methods, including:
obtaining a first matching result by a first matching method;
obtaining a second matching result by a second matching method;
combining, linearly or non-linearly, the first matching result having a first weight and the second matching result having a second weight to derive a final matching result; and
matching the infrared image and the visible image based on the final matching result;

identify a target in the combined image; and
control at least one of the UAV, the carrier, or the imaging device to track the identified target.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Weng Chao
• Zhang Wei
• Wang Mingxi

Classifications

• B64U2101/60

  for transporting passengers; for transporting goods other than weapons · CPC title

• B64U2101/16

  for controlling, capturing or immobilising other vehicles · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06T7/33 Primary

  using feature-based methods · CPC title