Aerial vehicle descent with delivery location identifiers

Patent metadata

Field	Value
Publication number	US-10871702-B2
Application number	US-201816110073-A
Country	US
Kind code	B2
Filing date	Aug 23, 2018
Priority date	Sep 24, 2015
Publication date	Dec 22, 2020
Grant date	Dec 22, 2020

Abstract

Described are systems and methods for surveying a destination as an unmanned aerial vehicle (“UAV”) descends toward the destination. To confirm that the destination is clear of objects and includes a safe landing or delivery location, such as a substantially planar surface, the UAV may capture and process images at different altitudes during the descent. Feature points of a first image captured at a first altitude may be paired with feature points of a second image captured at a second, different altitude. A homography may be computed to confirm that the paired feature points lie in the same plane and then the two images may be registered based on the paired feature points. The registered images may then be processed to determine depth information and determine if descent of the UAV is to continue or be aborted.

First claim

What is claimed is:

1. A method, comprising:
obtaining, with a camera of an aerial vehicle at a first altitude above a destination, a first image of the destination, wherein the first image includes a first representation of a delivery location identifier;
subsequent to obtaining the first image, descending, with the aerial vehicle, in altitude toward the destination;
obtaining, with the camera of the aerial vehicle at a second altitude above the destination, a second image of the destination, wherein the second image includes a second representation of the delivery location identifier;
registering the first image and the second image, based at least in part on a plurality of feature points of the delivery location identifier detected in each of the first image and the second image;
generating a depth disparity map of the destination based at least in part on the registered first image and second image; and
determining, based at least in part on the depth disparity map, a plane at the destination;
determining that the plane includes an area of sufficient size to accept at least one of a package from the aerial vehicle or the aerial vehicle; and
continuing to descend, with the aerial vehicle, toward the destination.

2. The method of claim 1 , further comprising:
obtaining, with the camera of the aerial vehicle at a third altitude above the destination, a third image of the destination;
processing the second image and the third image to detect an object; and
in response to detecting the object, aborting descent of the aerial vehicle.

3. The method of claim 1 , further comprising:
determining that a target altitude of the aerial vehicle has been achieved; and
in response to determining that the target altitude has been achieved, complete a delivery of the package carried by the aerial vehicle.

4. A computer-implemented method, comprising:
under control of one or more computing systems configured with executable instructions,
causing an aerial vehicle to descend toward a destination;
causing an image capture device of the aerial vehicle to obtain a first image of the destination when the aerial vehicle is at a first position, wherein the first image includes a first representation of a delivery location identifier;
causing the image capture device of the aerial vehicle to obtain a second image of the destination when the aerial vehicle is at a second position that is different than the first position, wherein the second image includes a second representation of the delivery location identifier;
registering the first image and the second image, based at least in part on a plurality of feature points of the delivery location identifier detected in each of the first image and the second image;
generating a depth disparity map of the destination based at least in part on the registered first image and second image; and
determining, based at least in part on the depth disparity map, that a substantially planar surface is available at the destination; and
causing, in response to determining that the substantially planar surface is available at the destination, the aerial vehicle to continue to descend toward the destination.

5. The computer-implemented method of claim 4 , wherein the substantially planar surface determined to be available at the destination is of sufficient size to accept at least one of a package from the aerial vehicle or the aerial vehicle.

6. The computer-implemented method of claim 4 , wherein the plurality of feature points includes a first plurality of feature points of the first image; and
the computer-implemented method further comprising:
for each of the first plurality of feature points of the first image, associating a feature point of the first plurality of feature points with a corresponding feature point of the second image to form paired feature points that include the feature point of the first image and the corresponding feature point of the second image;
determining a first plurality of paired feature points in which the feature point of the first image and the corresponding feature point of the second image are in a same plane; and
wherein registering the first image and the second image is based at least in part on the first plurality of paired feature points.

7. The computer-implemented method of claim 6 , wherein determining a first plurality of paired feature points in which the feature point of the first image and the corresponding feature point of the second image are in the same plane, further includes:
for each paired feature point:
determining a homography between the feature point of the first image and the corresponding feature point of the second image.

8. The computer-implemented method of claim 4 , further comprising one or more of:
processing a third image to determine a visual identifier included at the destination;
detecting a radio frequency identification tag at the destination; or
detecting a sound emitted from the destination.

9. The computer-implemented method of claim 4 , wherein generating a depth disparity map includes:
applying a stereo matching algorithm to the first image and the second image to generate the depth disparity map.

10. The computer-implemented method of claim 4 , wherein processing the first image and the second image includes:
defining a plane within the destination; and
determining that the substantially planar surface is within the plane.

11. The computer-implemented method of claim 4 , further comprising:
determining that the aerial vehicle has completed descent toward the destination; and
causing the aerial vehicle to release a payload at the destination, wherein the payload includes an item ordered from an electronic commerce website.

12. The computer-implemented method of claim 4 , further comprising:
causing the image capture device of the aerial vehicle to obtain a third image of the destination when the aerial vehicle is in a third position;
processing the third image to detect an object; and
in response to detecting the object, aborting descent of the aerial vehicle.

13. The computer-implemented method of claim 12 , further comprising:
determining that the object is within a defined distance of the delivery location identifier.

14. A computing system, comprising:
one or more processors; and
a memory coupled to the one or more processors and storing program instructions that, when executed by the one or more processors, cause the one or more processors to at least:
cause an aerial vehicle to descend toward a destination;
cause a camera of the aerial vehicle to obtain a first image of the destination when the aerial vehicle is at a first position, wherein the first image includes a first representation of a delivery location identifier;
cause the camera of the aerial vehicle to obtain a second image of the destination when the aerial vehicle is at a second position, wherein the second image includes a second representation of the delivery location identifier;
register the first image and the second image, based at least in part on a plurality of feature points of the delivery location identifier detected in each of the first image and the second image;
generate a depth disparity map of the destination based at least in part on the registered first image and second image; and
determine, based at least in part on the depth disparity map, that a substantially planar surface is available at the destination; and
cause, in response to a determination that the substantially planar surface is available at the destination, the aerial vehicle to continue to descend toward the destination.

Assignees

• Amazon Tech Inc

Inventors

• Harris Scott Raymond
• Safarik Stephen Thomas
• Viola Paul

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• G03B15/006 Primary

  Apparatus mounted on flying objects · CPC title

• G06V10/40

  Extraction of image or video features · CPC title

• G06V20/13

  Satellite images · CPC title

• G05D1/0676 Primary

  specially adapted for landing · CPC title