Object detection and avoidance for aerial vehicles

Patent metadata

Field	Value
Publication number	US-10402646-B2
Application number	US-201715838250-A
Country	US
Kind code	B2
Filing date	Dec 11, 2017
Priority date	Sep 21, 2017
Publication date	Sep 3, 2019
Grant date	Sep 3, 2019

Abstract

Aerial vehicles that are equipped with one or more imaging devices may detect obstacles that are small in size, or obstacles that feature colors or textures that are consistent with colors or textures of a landing area, using pairs of images captured by the imaging devices. Disparities between pixels corresponding to points of the landing area that appear within each of a pair of the images may be determined and used to generate a reconstruction of the landing area and a difference image. If either the reconstruction or the difference image indicates the presence of one or more obstacles, a landing operation at the landing area may be aborted or an alternate landing area for the aerial vehicle may be identified accordingly.

First claim

What is claimed is:

1. A method for detecting obstructions in a landing area for an unmanned aerial vehicle, the method comprising:
determining, by at least one sensor provided on the unmanned aerial vehicle, that an altitude of the unmanned aerial vehicle is at or below a predetermined altitude threshold;
locating, by at least one imaging device provided on the unmanned aerial vehicle, a target marker on a surface beneath the unmanned aerial vehicle;
defining the landing area based at least in part on at least a portion of the target marker, wherein the landing area comprises a geometric shape defined with respect to the portion of the target marker;
capturing, by the at least one imaging device, a first image including at least a portion of the surface beneath the unmanned aerial vehicle;
capturing, by the at least one imaging device, a second image including at least a portion of the surface beneath the unmanned aerial vehicle;
determining disparities between pixels corresponding to at least a plurality of points depicted in the first image and pixels corresponding to at least the plurality of points depicted in the second image, wherein each of the plurality of points is within the landing area;
generating a reconstruction of the landing area based at least in part on the disparities between the pixels;
calculating a difference image based at least in part on the disparities between the pixels;
determining whether the landing area includes at least one obstruction based at least in part on the reconstruction or the difference image; and
in response to determining that the landing area includes the at least one obstruction,
aborting a landing of the unmanned aerial vehicle at the landing area.

2. The method of claim 1 , wherein the at least one imaging device comprises a first imaging device and a second imaging device,
wherein the first imaging device and the second imaging device are separated by a baseline distance,
wherein the first image is captured by the first imaging device,
wherein the second image is captured by the second imaging device, and
wherein generating the reconstruction of the landing area comprises:
calculating, for each of the plurality of points, a distance between one of the pixels corresponding to the plurality of points depicted in the first image to one of the pixels corresponding to the plurality of points depicted in the second image,

wherein the disparities are determined based at least in part on the distances calculated for each of the plurality of points.

3. The method of claim 1 , wherein determining the disparities comprises:
providing each of the first image and the second image to an optical flow algorithm as inputs; and
receiving an output from the optical flow algorithm, wherein the output comprises a vector field comprising a plurality of vectors, and
wherein each of the vectors represents a disparity between one of the pixels corresponding to the plurality of points depicted in the first image and one of the pixels corresponding to the plurality of points depicted in the second image.

4. The method of claim 1 , wherein determining the disparities comprises:
matching each of the pixels corresponding to the plurality of points depicted in the first image to one of the pixels corresponding to the plurality of points depicted in the second image; and
determining, for each of the pixels corresponding to the plurality of points depicted in the first image, a distance to the matched one of the pixels corresponding to the plurality of points depicted in the second image,
wherein each of the distances calculated for each of the plurality of points is one of the disparities.

5. A method comprising:
capturing a first image by at least one imaging device provided aboard an aerial vehicle during the performance of an evolution by the aerial vehicle, wherein a field of view of the least one imaging device includes at least one surface at a first location;
capturing a second image by the at least one imaging device during the performance of the evolution by the aerial vehicle;
defining an operating area with respect to at least one point on the at least one surface;
determining a first plurality of pixel disparities between a first plurality of pixels of the first image and a second plurality of pixels of the second image, wherein at least some of the first plurality of pixels of the first image correspond to points of the operating area depicted within the first image, and wherein at least some of the second plurality of pixels correspond to the points of the operating area depicted within the second image;
generating a first three-dimensional reconstruction of at least a portion of the operating area based at least in part on the first plurality of pixel disparities;
generating a first difference image based at least in part on the first plurality of pixel disparities;
determining whether the operating area includes at least one obstruction based at least in part on at least the first three-dimensional reconstruction or the first difference image; and
in response to determining that the operating area includes the at least one obstruction,
suspending the evolution of the aerial vehicle.

6. The method of claim 5 , further comprising:
in response to determining that the operating area does not include the at least one obstruction,
completing the evolution of the aerial vehicle.

7. The method of claim 5 , wherein determining the first plurality of pixel disparities comprises:
providing each of the first image and the second image to an optical flow algorithm as inputs; and
receiving an output from the optical flow algorithm, wherein the output comprises a vector field comprising a plurality of vectors, and
wherein each of the plurality of vectors represents one of the pixel disparities.

8. The method of claim 5 , wherein determining the first plurality of pixel disparities comprises:
matching each of the first plurality of pixels of the first image to one of the second plurality of pixels of the second image; and
determining, for each of the first plurality of pixels, a distance to the matched one of the second plurality of pixels of the second image,
wherein each of the distances corresponds to one of the pixel disparities.

9. The method of claim 5 , wherein the at least one imaging device comprises a first imaging device and a second imaging device,
wherein the first imaging device and the second imaging device are separated by a baseline distance,
wherein the first image is captured by the first imaging device,
wherein the second image is captured by the second imaging device, and
wherein generating the first three-dimensional reconstruction of the operating area comprises:
calculating, for each of the points of the operating area, a distance from one of the first imaging device or the second imaging device to the one of the points; and
generating the first three-dimensional reconstruction of the operating area based at least in part on the distances from the one of the first imaging device or the second imaging device and the points of the operating area.

10. The method of claim 5 , wherein the at least one imaging device comprises a first imaging device,
wherein the first image is captured by the first imaging device at a first time,
wherein the second image is captured by the first imaging device at a second time.

11. The method of claim 5 , wherein the first image is captured at a first time and the second image is captured at approximately the first time, and
wherein the method further comprises:
in response to determining that the operating area does not include the at least one ob

Assignees

• Amazon Tech Inc

Inventors

• Klaus Andreas

Classifications

• G06Q10/083 Primary

  Shipping · CPC title

• G06V20/17

  taken from planes or by drones · CPC title

• G06V20/13

  Satellite images · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title