System and method for supporting smooth target following

Patent metadata

Field	Value
Publication number	US-12181879-B2
Application number	US-202117224391-A
Country	US
Kind code	B2
Filing date	Apr 7, 2021
Priority date	Sep 15, 2015
Publication date	Dec 31, 2024
Grant date	Dec 31, 2024

Abstract

A method for controlling target tracking includes determining whether a tracking mode of a movable object is a manual tracking mode or an automatic tracking mode; when it is determined that the tracking mode is the manual tracking mode, determining a target for the movable object from one or more images based on a user input; when it is determined that the tracking mode is the automatic tracking mode, having the movable object determine the target automatically; and directing the movable object to track a target object using the target.

First claim

What is claimed is:

1. A method for controlling target tracking, comprising:
determining a target for a movable object to track in one or more images captured by the movable object;
directing the movable object to track a target object using the target; and
in response to a loss of the target in the one or more images during tracking, controlling; the movable object to redetect the target using one or more subsequent images captured by the movable object to continue tracking the target object, including:
adjusting both the movable object and an imaging device carried by the movable object in a same axis to redetect the target using the one or more subsequent images captured by the imaging device to continue tracking the target object.

2. The method of claim 1 , wherein the determining a target for a movable object to track in one or more images captured by the movable object further comprises:
determining the target for the movable object based on a user input; or
having the movable object determine the target automatically.

3. The method of claim 2 , wherein the having the movable object determine the target automatically further comprises:
obtaining target type information indicative of a type of target object to be tracked; and
providing the target type information to the movable object to automatically identify the target based on the target type information.

4. The method of claim 3 , wherein the target type information is obtained via a user interface provided by a control terminal remotely coupled to the movable object via a communication link.

5. The method of claim 1 , wherein the directing the movable object to track a target object using the target further comprises:
generating control data based on the target and sensing data provided by the movable object; and
generating one or more control signals to direct the movable object to track the target object,
wherein the control data is generated by at least one of a control terminal remotely coupled to the movable object via a communication link or an onboard processing system of the movable object.

6. The method of claim 5 , wherein:
an allocation between the control terminal and the onboard processing system is determined in real time based on a plurality of factors including environment information of the movable object.

7. The method of claim 5 , wherein:
when the movable object is navigating in a relatively complex environment, the control terminal is allocated to generate more control data than the onboard processing system of the movable object; and
when the movable object is navigating in a relatively simple environment, the control terminal is allocated to generate less control data than the onboard processing system of the movable object.

8. The method of claim 5 , wherein:
when the movable object is navigating in a relatively low altitude, the control terminal is allocated to generate more control data than the onboard processing system of the movable object; and
when the movable object is navigating in a relatively high altitude, the control terminal is allocated to generate less control data than the onboard processing system of the movable object.

9. The method of claim 5 , wherein the control data includes at least a relative distance between the movable object and the target object.

10. The method of claim 5 , wherein the control data includes at least a similarity between a feature model of the target object and features from the target.

11. The method of claim 1 , wherein controlling the movable object to redetect the target using the one or more subsequent images captured by the movable object to continue tracking the target object further includes:
in response to detecting that the movable object is in an indoor environment, controlling the movable object to use vision based tracking to redetect the target only using the one or more subsequent images captured by the movable object; or
in response to detecting that the movable object is in an outdoor environment, controlling the movable object to locate the target object using a positioning device carried by the target object for redetecting the target using one or more subsequent images captured by the movable object to continue tracking the target object.

12. The method of claim 11 , wherein:
the positioning device carried by the target object is a first positioning device; and
the controlling the movable object to locate the target object further comprises:
controlling the movable object to locate the target object according to a relative distance and/or orientation calculated based on sensor data obtained from the first positioning device carried by the target object and a second positioning device on the movable object.

13. A system for controlling target tracking, comprising:
one or more processors; and
one or more memories coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:
determine a target for a movable object to track in one or more images captured by the movable object;
direct the movable object to track a target object using the target; and
in response to a loss of the target in one or more images during tracking, control the movable object to redetect the target using one or more subsequent images captured by the movable object to continue tracking the target object, including:
adjusting both the movable object and an imaging device carried by the movable object in a same axis to redetect the target using the one or more subsequent images captured by the imaging device to continue tracking the target object.

14. The system of claim 13 , wherein the instructions are further executed by the one or more processors to cause the one or more processors to:
determine the target for the movable object based on a user input; or
have the movable object determine the target automatically.

15. The system of claim 14 , wherein the instructions are further executed by the one or more processors to cause the one or more processors to:
obtain target type information indicative of a type of target object to be tracked; and
provide the target type information to the movable object to automatically identify the target based on the target type information.

16. The system of claim 15 , wherein the target type information is obtained via a user interface provided by a control terminal remotely coupled to the movable object via a communication link.

17. The system of claim 13 , wherein the instructions are further executed by the one or more processors to cause the one or more processors to:
generate control data based on the target and sensing data provided by the movable object; and
generate one or more control signals to direct the movable object to track the target object,
wherein the control data is generated by at least one of the control terminal or an onboard processing system of the movable object.

18. The system of claim 17 , wherein:
when the movable object is navigating in a relatively complex environment, the control terminal is allocated to generate more control data than the onboard processing system of the movable object; and
when the movable object is navigating in a relatively simple environment, the control terminal is allocated to generate less control data than the onboard processing system of the movable object.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Zhao Cong
• Pang Minjian
• Li Rui
• Zhou You
• Liu Zhe
• Zhou Guyue

Classifications

• G05D1/221

  Remote-control arrangements · CPC title

• G06V10/62

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title