Methods and system for infrared tracking

Patent metadata

Field	Value
Publication number	US-11526998-B2
Application number	US-201916728448-A
Country	US
Kind code	B2
Filing date	Dec 27, 2019
Priority date	Nov 3, 2017
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

A computer-implemented method for tracking includes obtaining an infrared image and a visible image from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), obtaining a combined image based on the infrared image and the visible image, identifying a target in the combined image, and generating control signals for tracking the identified target using the imaging device.

First claim

What is claimed is:

1. A method for tracking, comprising:
obtaining, from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), an infrared image and a visible image;
obtaining a combined image based on the infrared image and the visible image, including:
applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image; and

controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;
wherein:
the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;
the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and
the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

2. The method of claim 1 , wherein obtaining the combined image further comprises:
processing the infrared image to extract infrared features;
processing the visible image to extract the visible features;
matching the infrared image and the visible image based on the infrared features and the visible features; and
applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

3. The method of claim 2 , wherein the infrared features comprise an infrared outer contour and the visible features comprise a visible outer contour, and wherein matching the infrared image and the visible image comprises aligning the infrared outer contour with the visible outer contour.

4. The method of claim 1 , wherein identifying the target in the combined image comprises obtaining target information from a remote terminal and identifying the target based on the target information.

5. The method of claim 1 , wherein generating the control signals for tracking the identified target comprises determining whether to control the UAV, the carrier, or the imaging device based on a current configuration of the UAV, the carrier, or the imaging device.

6. The method of claim 1 , wherein generating the control signals for tracking the identified target comprises detecting a deviation of the target from a predetermined configuration and generating the control signals for substantially correcting the deviation.

7. An unmanned aerial vehicle (UAV), comprising:
a carrier;
a memory that stores one or more computer-executable instructions; and
one or more processors configured to access the memory and execute the computer-executable instructions to perform a method comprising:
obtaining, from an imaging device supported by the carrier, an infrared image and a visible image;
obtaining a combined image based on the infrared image and the visible image, including:
applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image;

identifying a target in the combined image; and
controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;
wherein:
the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;
the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and
the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

8. The UAV of claim 7 , wherein obtaining the combined image comprises:
processing the infrared image to extract infrared features;
processing the visible image to extract the visible features;
matching the infrared image and the visible image based on the infrared features and the visible features; and
applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

9. The UAV of claim 8 , wherein the infrared features comprise an infrared outer contour and the visible features comprise a visible outer contour, and wherein matching the infrared image and the visible image comprises aligning the infrared outer contour with the visible outer contour.

10. The UAV of claim 7 , wherein generating the control signals for tracking the identified target comprises determining whether to control the UAV, the carrier, or the imaging device based on a current configuration of the UAV, the carrier, or the imaging device.

11. The UAV of claim 7 , wherein generating the control signals for tracking the identified target comprises detecting a deviation of the target from a predetermined configuration and generating the control signals for substantially correcting the deviation.

12. A tracking system, comprising:
a memory that stores one or more computer-executable instructions; and
one or more processors configured to access the memory and execute the computer-executable instructions to perform a method comprising:
obtaining, from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), an infrared image and a visible image;
obtaining a combined image based on the infrared image and the visible image, including:
applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image;

identifying a target in the combined image; and
controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;
wherein:
the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;
the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and
the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

13. The system of claim 12 , wherein obtaining the combined image comprises:
processing the infrared image to extract infrared features;
processing the visible image to extract the visible features;
matching the infrared image and the visible image based on the infrared features and the visible features; and
applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Weng Chao
• Zhang Wei
• Wang Mingxi

Classifications

• B64U2201/20

  Remote controls · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T2207/10036

  Multispectral image; Hyperspectral image · CPC title