Methods and system for vision-based landing

Patent metadata

Field	Value
Publication number	US-11604479-B2
Application number	US-201916365062-A
Country	US
Kind code	B2
Filing date	Mar 26, 2019
Priority date	Sep 26, 2016
Publication date	Mar 14, 2023
Grant date	Mar 14, 2023

Abstract

A computer-implemented method for controlling an unmanned aerial vehicle (UAV) includes obtaining a first image captured by an imaging device carried by the UAV during a takeoff of the UAV from a target location, obtaining a second image from the imaging device in response to an indication to return to the target location, determining a spatial relationship between the UAV and the target location by comparing the first image and the second image, and controlling the UAV to approach the target location based at least in part on the spatial relationship.

First claim

What is claimed is:

1. A computer-implemented method for controlling an unmanned aerial vehicle (UAV), comprising:
obtaining a plurality of first images captured by an imaging device carried by the UAV during a takeoff of the UAV from a target location, the plurality of first images being captured with a spatial or temporal interval dynamically determined based in part on an environment surrounding the UAV including terrain information surrounding the UAV;
in response to an indication to return to the target location, obtaining a second image from the imaging device;
determining a spatial relationship between the UAV and the target location by comparing the second image and a corresponding one of the plurality of first images; and
controlling the UAV to approach the target location based at least in part on the spatial relationship.

2. The method of claim 1 , wherein the plurality of first images and the second image are obtained during a same flight of the UAV.

3. The method of claim 1 , wherein the corresponding one of the plurality of first images or the second image does not include the target location.

4. The method of claim 1 , wherein the indication to return to the target location is received from a remote device or generated by one or more processors onboard the UAV.

5. The method of claim 1 , wherein the corresponding one of the plurality of first images is captured at a first pose and the second image is captured at a second pose, and wherein determining the spatial relationship between the UAV and the target location comprises:
determining a transformation between the first pose and the second pose based on the comparison between the second image and the corresponding one of the plurality of first images; and
determining the second pose based at least in part on the first pose and the transformation.

6. The method of claim 5 , wherein the first pose comprises a position or an orientation of the UAV when the corresponding one of the plurality of first images is captured.

7. The method of claim 5 , wherein the first pose comprises a position or an orientation of the imaging device when the corresponding one of the plurality of first images is captured.

8. The method of claim 5 , wherein the corresponding one of the plurality of first images is captured at a first orientation of the imaging device, and wherein the method further comprises causing the imaging device to have substantially the first orientation before the second image is captured.

9. The method of claim 5 , wherein:
the first pose is determined based at least in part on a measurement from a non-vision position sensor; and
the second pose is determined without using the measurement from the non-vision position sensor.

10. The method of claim 1 , wherein:
the imaging device is controlled to face a non-downward direction during an initial takeoff and is controlled to rotate towards a downward direction in response to the UAV ascending during the takeoff of the UAV; and
the imaging device is controlled to face the downward direction during an initial return and is controlled to rotate towards the non-downward direction in response to the UAV descending during the return of the UAV.

11. The method of claim 1 , wherein the spatial or temporal interval is dynamically determined based further in part on at least one of:
a state of the UAV including at least one of an altitude, a processing power, a battery life, or an available storage space of the UAV, or
a state of the imaging device including at least one of an altitude, a processing power, a battery life, or an available storage space of the imaging device.

12. An unmanned aerial vehicle (UAV), comprising:
a memory that stores one or more computer-executable instructions; and
one or more processors configured to access the memory and execute the computer-executable instructions to perform a method comprising:
obtaining a plurality of first images captured by an imaging device carried by the UAV during a takeoff of the UAV from a target location, the plurality of first images being captured with a spatial or temporal interval dynamically determined based in part on an environment surrounding the UAV, the environment surrounding the UAV including terrain information surrounding the UAV;
in response to an indication to return to the target location, obtaining a second image from the imaging device;
determining a spatial relationship between the UAV and the target location by comparing the second image and a corresponding one of the plurality of first images; and
controlling the UAV to approach the target location based at least in part on the spatial relationship.

13. The UAV of claim 12 , wherein the plurality of first images and the second image are obtained during a same flight of the UAV.

14. The UAV of claim 12 , wherein the corresponding one of plurality of first images or the second image does not include the target location.

15. The UAV of claim 12 , wherein the indication to return to the target location is received from a remote device or generated by one or more processors onboard the UAV.

16. The UAV of claim 12 , wherein the corresponding one of the plurality of first images is captured at a first pose and the second image is captured at a second pose, and wherein determining the spatial relationship between the UAV and the target location comprises:
determining a transformation between the first pose and the second pose based on the comparison between the second image and the corresponding one of the plurality of first images; and
determining the second pose based at least in part on the first pose and the transformation.

17. The UAV of claim 16 , wherein the first pose comprises a position or an orientation of the UAV when the corresponding one of the plurality of first images is captured.

18. The UAV of claim 16 , wherein the first pose comprises a position or an orientation of the imaging device when the corresponding one of the plurality of first images is captured.

19. A computer-implemented method for determining a position of an unmanned aerial vehicle (UAV), comprising:
determining a plurality of first positions of the UAV based at least in part on a measurement from a non-vision position sensor carried by the UAV;
associating the plurality of first positions of the UAV with a plurality of first images captured by an imaging device carried by the UAV when the UAV is at a plurality of first poses, each of the plurality of first poses including a corresponding one of the plurality of the first positions, the plurality of first images being captured with a spatial or temporal interval dynamically determined based in part on an environment surrounding the UAV, the environment surrounding the UAV including terrain information surrounding the UAV; and
determining a second position of the UAV by comparing a second image with a corresponding one of the plurality of first images, the second image being captured by the imaging device when the UAV is at a second pose, the second pose including the second position and being determined without using the measurement of the non-vision position sensor.

Assignees

• Sz Dji Technology Co Ltd

Inventors

• Wu Bo
• Liu Ang
• Zhang Litian

Classifications

• B64U2101/30

  for imaging, photography or videography · CPC title

• G05D1/0676 Primary

  specially adapted for landing · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64D47/08

  Arrangements of cameras · CPC title

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title