Maximum temperature point tracking method, device and unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-11153494-B2
Application number	US-201916728383-A
Country	US
Kind code	B2
Filing date	Dec 27, 2019
Priority date	Nov 30, 2017
Publication date	Oct 19, 2021
Grant date	Oct 19, 2021

Abstract

In the method, a first pair of coordinates of a maximum temperature point in an image sensed by an infrared camera is acquired, where the image is an image captured by the infrared camera. A rotation angle of a gimbal equipped with the infrared camera is then determined according to the first pair of coordinates of the maximum temperature point and a pair of coordinates of a target position in the image. The gimbal is then controlled to rotate according to the determined rotation angle, so as to adjust the maximum temperature point in the image captured by the infrared camera to be located at the target position.

First claim

What is claimed is:

1. A method for tracking a maximum temperature point, comprising:
acquiring a first pair of coordinates of a maximum temperature point sensed by an infrared camera in a current frame of image, the current frame of image being an image captured by the infrared camera;
obtaining a temperature of the maximum temperature point sensed by the infrared camera in the current frame of image;
predicting an expected pair of coordinates of the maximum temperature point in a subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point and the first pair of coordinates;
determining a rotation angle of a gimbal equipped with the infrared camera according to the expected pair of coordinates and a pair of coordinates of a target position in the subsequent frame of image; and
controlling the gimbal to rotate according to the rotation angle, so as to adjust the maximum temperature point in the subsequent frame of image captured by the infrared camera to be located at the target position.

2. The method according to claim 1 , wherein predicting the expected pair of coordinates of the maximum temperature point in the subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point and the first pair of coordinates further includes:
when the temperature of the maximum temperature point in the current frame of image falls within a maximum temperature confidence interval, determining the first pair of coordinates of the maximum temperature point in the current frame of image to be the expected pair of coordinates; or
when the temperature of the maximum temperature point in the current frame of image falls within the maximum temperature confidence interval, determining the expected pair of coordinates according to the first pair of coordinates of the maximum temperature point in the current frame of image and first pairs of coordinates of maximum temperature points of preceding N frames of images,
wherein the preceding N frames of images are consecutive N frames of images in which temperatures of the maximum temperature points fall within the maximum temperature confidence interval, and N is an integer greater than or equal to 1.

3. The method according to claim 1 , wherein, before predicting the expected pair of coordinates of the maximum temperature point in the subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point and the first pair of coordinates, the method further includes:
determining a second pair of coordinates of an expected maximum temperature point in the current frame of image captured by the infrared camera according to visual tracking; and
wherein predicting the expected pair of coordinates of the maximum temperature point in the subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point and the first pair of coordinates further includes:
predicting the expected pair of coordinates of the maximum temperature point in the subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point, the first pair of coordinates, and the second pair of coordinates.

4. The method according to claim 3 , wherein, before determining the second pair of coordinates of the expected maximum temperature point in the current frame of image captured by the infrared camera according to visual tracking, the method further includes:
acquiring a bitstream feature at the expected maximum temperature point; and
wherein determining the second pair of coordinates of the maximum temperature point in the current frame of image captured by the infrared camera according to visual tracking further includes:
acquiring an infrared bitstream of the current frame of image captured by the infrared camera, and
visually tracking the infrared bitstream according to the bitstream feature to determine the second pair of coordinates.

5. The method according to claim 4 , wherein acquiring the bitstream feature of the expected maximum temperature point further includes:
determining the bitstream feature of the expected maximum temperature point according to bitstream features of maximum temperature points corresponding to preceding M expected pairs of coordinates determined, wherein M is an integer greater than or equal to 1.

6. The method according to claim 3 , wherein predicting the expected pair of coordinates of the maximum temperature point in the subsequent frame of image to be captured by the infrared camera according to the temperature of the maximum temperature point, the first pair of coordinates, and the second pair of coordinates further includes:
when the temperature of the maximum temperature point in the current frame of image falls within the maximum temperature confidence interval, determining the expected pair of coordinates according to the first pair of coordinates of the maximum temperature point in the current frame of image and the second pair of coordinates of the expected maximum temperature point in the current frame of image; or
when the temperature of the maximum temperature point in the current frame of image falls within the maximum temperature confidence interval, determining the expected pair of coordinates according to the first pair of coordinates of the maximum temperature point in the current frame of image, the second pair of coordinates of the expected maximum temperature point in the current frame of image, first pairs of coordinates of maximum temperature points in preceding N frames of images, and second pairs of coordinates of expected maximum temperature points in the preceding N frames of images,
wherein the preceding N frames of images are consecutive N frames of images in which temperatures of the maximum temperature points fall within the maximum temperature confidence interval, and N is an integer greater than or equal to 1.

7. The method according to claim 2 , further comprising:
when the temperature of the maximum temperature point in the current frame of image does not fall within the maximum temperature confidence interval, determining an expected pair of coordinates determined in a preceding frame of image to be the expected pair of coordinates.

8. The method according to claim 2 , wherein determining whether the temperature of the maximum temperature point in the current frame of image falls within the maximum temperature confidence interval further includes:
when a variance or standard deviation between the temperature of the maximum temperature point in the current frame of image and temperatures of maximum temperature points in preceding K frames of images is less than a preset value, determining that the maximum temperature point in the current frame of image falls within the maximum temperature confidence interval,
wherein the preceding K frames of images are consecutive K frames of images in which temperatures of the maximum temperature points fall within the maximum temperature confidence interval, and K is an integer greater than or equal to 1.

9. The method according to claim 2 , wherein determining the expected pair of coordinates according to the first pair of coordinates of the maximum temperature point in the current frame of image and the first pairs of coordinates of the maximum temperature points of the preceding N frames of images includes:
performing an operation of a least squares method on the first pair of coordinates of the maximum temperature point in the current frame of image and the first pairs of coordinates of the maximum temperature points of the preceding

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Weng Chao
• Wang Mingxi
• Zhang Wei

Classifications

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06V20/176

  Urban or other man-made structures · CPC title

• G06V10/143

  Sensing or illuminating at different wavelengths · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• G01J5/00

  Radiation pyrometry, e.g. infrared or optical thermometry · CPC title