Systems and methods for UAV interactive instructions and control

Patent metadata

Field	Value
Publication number	US-11635775-B2
Application number	US-202016991739-A
Country	US
Kind code	B2
Filing date	Aug 12, 2020
Priority date	Sep 15, 2015
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

A method for controlling an unmanned aerial vehicle (UAV) includes receiving, by a processor of the UAV, a plurality of images captured by an imaging device coupled to the UAV, identifying, by the processor, a target in at least one image of the plurality of images, determining, by the processor, whether the target is a stationary target or a moving target based on analyzing the plurality of images, and automatically effecting, by the processor, movement of the UAV based on determining the target is the stationary target or the moving target.

First claim

What is claimed is:

1. A method for controlling an unmanned aerial vehicle (UAV), comprising:
receiving, by a processor of the UAV, a plurality of images captured by an imaging device coupled to the UAV;
receiving, by the processor of the UAV, an input signal indicating user selection of a point or a portion in at least one image of the plurality of images, the input signal including information of the point or the portion in the at least one image that is associated with a target;
identifying, by the processor, the target in the at least one image of the plurality of images based on the input signal and feature point recognition;
determining, by the processor, whether the target is a stationary target or a moving target based on analyzing the plurality of images to track feature points of the identified target in the plurality of images;
automatically effecting, by the processor, movement of the UAV based on determining the target is the stationary target or the moving target; and
adjusting motion characteristics of the UAV while tracking the target according to at least one of (1) a number of times a user selects the point or the portion in the at least one image or (2) a time duration when the user selects the point or the portion in the at least one image, wherein the motion characteristics include at least one of a velocity, an acceleration, or an orientation of the UAV.

2. The method of claim 1 , wherein the feature point recognition comprises:
extracting one or more features points from the at least one image using one or more algorithms, wherein the one or more algorithms comprise at least one of texture detection algorithm, algorithm of calculating a total number of the one or more feature points, algorithm of calculating a distribution of the one or more feature points, or algorithm of calculating a quality of the one or more feature points based on at least one of edge detection, corner detection, blob detection, or ridge detection.

3. The method of claim 1 , wherein determining whether the target is the stationary target or the moving target comprises:
determining relative positions or moving directions between the target and the imaging device or between the target and the UAV;
determining a position of the imaging device or the UAV with respect to an environment;
determining a position of the target with respect to the environment; or
determining positional information of the imaging device, the UAV, or the target based on sensing data received from one or more additional sensors coupled to the UAV.

4. The method of claim 1 , wherein automatically effecting movement of the UAV comprises:
automatically switching a mode of the UAV between a plurality of modes based on determining the target is the stationary target or the moving target; and
effecting movement of the UAV based on the mode.

5. The method of claim 4 , wherein the plurality of modes comprise at least a target mode and a directional mode, wherein the UAV is configured to move towards or follow the target when the mode is the target mode and the UAV is configured to move in a specific direction when the mode is the directional mode.

6. The method of claim 5 , wherein the target mode comprises a fly-to mode and the method further comprises:
automatically switching the mode of the UAV to the fly-to mode when the target is determined to be the stationary target or when a relatively direct path exists between the UAV and the target.

7. The method of claim 5 , wherein the target mode comprises a tracking mode and the method further comprises:
automatically switching the mode of the UAV to the tracking mode when the target is determined to be the moving target or when a flight path exists between the UAV and the target that a clear line of sight is absent between the UAV and the target at one or more portions of the flight path.

8. The method of claim 5 , wherein the target mode comprises a fly-to mode and the method further comprises:
automatically switching the mode of the UAV to the fly-to mode when fewer than a predetermined number of obstacles and/or when fewer than a predetermined number of types of obstacles are determined to be present as the UAV is moving toward the target; and
directing the UAV to move toward the target when the mode of the UAV is the fly-to mode, wherein the UAV is moved to a predetermined distance from the target.

9. The method of claim 5 , wherein the target mode comprises a tracking mode and the method further comprises:
automatically switching the mode of the UAV to the tracking mode when greater than a predetermined number of obstacles and/or when greater than a predetermined number of types of obstacles are determined to be present as the UAV is moving toward the target; and
directing the UAV to follow the target at a predetermined distance when the mode of the UAV is the tracking mode.

10. A non-transitory computer-readable medium storing instructions that, when executed, cause a computer to perform a method for controlling an unmanned aerial vehicle (UAV), the method comprising:
receiving, by a processor of the UAV, a plurality of images captured by an imaging device coupled to the UAV;
receiving, by the processor, an input signal indicating user selection of a point or a portion in at least one image of the plurality of images, the input signal including information of the point or the portion in the at least one image that is associated with a target;
identifying, by the processor, the target in the at least one image of the plurality of images based on the input signal and feature point recognition;
determining, by the processor, whether the target is a stationary target or a moving target based on analyzing the plurality of images to track feature points of the identified target in the plurality of images;
automatically effecting, by the processor, movement of the UAV based on determining the target is the stationary target or the moving target; and
adjusting motion characteristics of the UAV while tracking the target according to at least one of (1) a number of times a user selects the point or the portion in the at least one image or (2) a time duration when the user selects the point or the portion in the at least one image, wherein the motion characteristics include at least one of a velocity, an acceleration, or an orientation of the UAV.

11. The non-transitory computer-readable medium of claim 10 , wherein determining whether the target is the stationary target or the moving target comprises:
determining relative positions or moving directions between the target and the imaging device or between the target and the UAV;
determining a position of the imaging device or the UAV with respect to an environment;
determining a position of the target with respect to the environment; or
determining positional information of the imaging device, the UAV, or the target based on sensing data received from one or more additional sensors coupled to the UAV.

12. The non-transitory computer-readable medium of claim 10 , wherein automatically effecting movement of the UAV comprises:
automatically switching a mode of the UAV between a plurality of modes based on determining the target is the stationary target or the moving target; and
effecting movement of the UAV based on the mode.

13. The non-transitory computer-readable medium of claim 12 , wherein the plurality of modes comprise at least a target mode and a directional mode, wherein the UAV is configured to move toward or follow the target when the mode is the target mode and the UAV is configured to move in a specific direction when the mode is the directional mode.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Wang Mingyu
• Zhao Tao
• Yu Yun
• Zhou Guyue
• Tang Ketan
• Zhao Cong
• Liu Ang
• Zhou You
• Pang Minjian
• Hu Xiao
• Zhang Litian

Classifications

• G05D1/221

  Remote-control arrangements · CPC title

• G06V10/62

  relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking · CPC title

• B64U2201/20

  Remote controls · CPC title

• B64U2101/31

  for surveillance · CPC title

• G05D1/0011

  associated with a remote control arrangement · CPC title