Determining stereo distance information using imaging devices integrated into propeller blades

Patent metadata

Field	Value
Publication number	US-10728516-B2
Application number	US-201816041441-A
Country	US
Kind code	B2
Filing date	Jul 20, 2018
Priority date	Aug 22, 2016
Publication date	Jul 28, 2020
Grant date	Jul 28, 2020

Abstract

A propeller provided on an aerial vehicle may include a digital camera or other imaging device embedded into a surface of one of the blades of the propeller. The digital camera may capture images while the propeller is rotating at an operational speed. Images captured by the digital camera may be processed to recognize one or more objects therein, and to determine ranges to such objects by stereo triangulation techniques. Using such ranges, a depth map or other model of the surface features in an environment in which the aerial vehicle is operating may be defined and stored or used for any purpose. A propeller may include digital cameras or other imaging devices embedded into two or more blades, and may also use such images to determine ranges to objects by stereo triangulation techniques.

First claim

What is claimed is:

1. An aerial vehicle comprising:
a frame;
a propulsion motor mounted to the frame, wherein the propulsion motor is configured to rotate a shaft about an axis defined by the shaft;
a propeller having a plurality of blades, wherein the propeller is rotatably coupled to the shaft;
a digital camera embedded in an underside of one of the plurality of blades; and
at least one computer processor,
wherein the at least one computer processor is configured to at least:
cause the propulsion motor to rotate the propeller about the axis at a predetermined speed;
cause the digital camera to capture a first digital image at a first time, wherein the one of the plurality of blades having the digital camera embedded therein is aligned in a first angular orientation at the first time;
cause the digital camera to capture a second digital image at a second time, wherein the one of the plurality of blades having the digital camera embedded therein is aligned in a second angular orientation at the second time;
determine a distance between a first position of the digital camera at the first time and a second position of the digital camera at the second time; and
determine an altitude of the aerial vehicle above at least a portion of a surface below the aerial vehicle based at least in part on the distance, a focal length of the digital camera, the first digital image and the second digital image.

2. The aerial vehicle of claim 1 , wherein the at least one computer processor is further configured to at least:
recognize a first representation of the portion of the surface below the aerial vehicle within at least a portion of the first image; and
recognize a second representation of the portion of the surface below the aerial vehicle within at least a portion of the second image,
wherein the altitude of the aerial vehicle is determined based at least in part on the distance, the focal length, the first representation and the second representation.

3. The aerial vehicle of claim 2 , wherein the at least one computer processor is further configured to at least:
define a first line extending from the first position of the imaging device through the first representation;
define a second line extending from the second position of the imaging device through the second representation; and
identify an intersection of the first line and the second line,
wherein the altitude of the aerial vehicle is determined based at least in part on the intersection of the first line and the second line.

4. The aerial vehicle of claim 1 , wherein the at least one computer processor is further configured to at least:
recognize a plurality of points of the portion of the surface below the aerial vehicle in the first digital image by the at least one computer processor;
recognize at least some of the plurality of points in the second digital image by the at least one computer processor;
determine ranges to the at least some of the plurality of points based at least in part on the distance, the focal length, the first digital image and the second digital image; and
generate a depth map based at least in part on the ranges to the at least some of the plurality of points,
wherein the altitude of the aerial vehicle is determined based at least in part on the depth map.

5. A method comprising:
capturing a first image of at least a portion of a scene by a first imaging device at a first time, wherein the first imaging device is integrated into a first surface of a first blade of a first propeller of a first aerial vehicle at a first radius from a first hub of the first propeller, and wherein the first propeller is rotating at a first angular velocity about a first axis of rotation at the first time;
capturing a second image of at least the portion of the scene by the first imaging device at a second time, wherein the first propeller is rotating at the first angular velocity about the first axis of rotation at the second time;
determining a distance between a first position of the first imaging device at the first time and a second position of the first imaging device at the second time; and
determining a first range to at least the portion of the scene based at least in part on the first image and the second image by at least one computer processor.

6. The method of claim 5 , wherein the first blade is in a first angular orientation at the first time,
wherein the first blade is in a second angular orientation at the second time, and
wherein the distance is determined based at least in part on the first angular orientation, the second angular orientation and the first radius.

7. The method of claim 5 , further comprising:
identifying a first representation of at least one point of the scene in the first image by the at least one computer processor; and
identifying a second representation of the at least one point of the scene in the second image by the at least one computer processor,
wherein determining the first range to at least the portion of the scene comprises:
defining a first line from the first position through the first representation by the at least one computer processor;
defining a second line from the second position through the second representation by the at least one computer processor;
determining a position of an intersection of the first line and the second line by the at least one computer processor; and
determining the first range to the portion of the scene based at least in part on the position of the intersection by the at least one computer processor.

8. The method of claim 5 , further comprising:
determining a first angular orientation of the first blade at the first time; and
determining a second angular orientation of the first blade at the second time,
wherein determining the first range to at least the portion of the scene comprises:
aligning the first image and the second image with respect to one another based on a difference between the first angular orientation and the second angular orientation.

9. The method of claim 5 , further comprising:
recognizing a plurality of points of the scene in the first image by the at least one computer processor;
recognizing at least some of the plurality of points of the scene in the second image by the at least one computer processor, and
wherein determining the first range to the at least one point of the scene comprises:
determining ranges to the at least some of the plurality of points of the scene based at least in part on the first image and the second image by the at least one computer processor;
defining a range map for the scene based at least in part on the ranges, wherein the range map represents distances to each of a plurality of regions; and
determining the first range based at least in part on the range map.

10. The method of claim 5 , further comprising:
selecting, by at least one computer processor, a landing site for the first aerial vehicle at the scene based at least in part on the first range.

11. The method of claim 5 , further comprising:
capturing a third image of at least the portion of the scene by a second imaging device at the first time or the second time, wherein the second imaging device is integrated into a second surface of a second blade of the first propeller at a second radius from the first hub; and
determining a second range to at least the portion of the scene based at least in part on the third image and at least one of the first image or the second image by the at least one computer processor.

12. The method of claim 5 , further comprising:
capturing a third image of at least the portion of the scene by a second imaging device at

Assignees

• Amazon Tech Inc

Inventors

• Boyd Scott Patrick
• O'Brien Barry James
• Watson Joshua John
• Wilcox Scott Michael

Classifications

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• H04N13/128 Primary

  Adjusting depth or disparity · CPC title

• G01C3/10 Primary

  using a parallactic triangle with variable angles and a base of fixed length in the observation station, e.g. in the instrument · CPC title

• B64U50/19

  using electrically powered motors · CPC title