Determining stereo distance information using imaging devices integrated into propeller blades

Patent metadata

Field	Value
Publication number	US-10033980-B2
Application number	US-201615243844-A
Country	US
Kind code	B2
Filing date	Aug 22, 2016
Priority date	Aug 22, 2016
Publication date	Jul 24, 2018
Grant date	Jul 24, 2018

Abstract

A propeller provided on an aerial vehicle may include a digital camera or other imaging device embedded into a surface of one of the blades of the propeller. The digital camera may capture images while the propeller is rotating at an operational speed. Images captured by the digital camera may be processed to recognize one or more objects therein, and to determine ranges to such objects by stereo triangulation techniques. Using such ranges, a depth map or other model of the surface features in an environment in which the aerial vehicle is operating may be defined and stored or used for any purpose. A propeller may include digital cameras or other imaging devices embedded into two or more blades, and may also use such images to determine ranges to objects by stereo triangulation techniques.

First claim

What is claimed is:

1. An unmanned aerial vehicle comprising:
a frame;
a first propulsion motor mounted to the frame, wherein the first propulsion motor is configured to rotate a first shaft about a first axis defined by the first shaft;
a first propeller having a first plurality of blades, wherein the first propeller is rotatably coupled to the first shaft;
a first digital camera embedded in an underside of a first one of the first plurality of blades; and
at least one computer processor,
wherein the at least one computer processor is configured to at least:
cause the first propulsion motor to rotate the first propeller at a first predetermined speed;
cause the first digital camera to capture a first digital image at a first time, wherein the first blade having the first digital camera embedded therein is aligned in a first angular orientation at the first time;
cause the first digital camera to capture a second digital image at a second time, wherein the first blade having the first digital camera embedded therein is aligned in a second angular orientation at the second time;
determine a baseline distance between a first position of the first digital camera at the first time and a second position of the first digital camera at the second time;
recognize a first representation of a portion of an object below the unmanned aerial vehicle within at least a portion of the first image;
recognize a second representation of the portion of the object below the unmanned aerial vehicle within at least a portion of the second image;
determine a first position of the portion of the object based at least in part on the baseline distance, a focal length of the first digital camera, the first representation and the second representation; and
determine a first distance to the portion of the object below the unmanned aerial vehicle based at least in part on the first position.

2. The unmanned aerial vehicle of claim 1 , wherein the at least one computer processor is further configured to at least:
define a first line extending from the first position of the imaging device through the first representation of the portion of the object;
define a second line extending from the second position of the imaging device through the second representation of the portion of the object; and
identify an intersection of the first line and the second line,
wherein the first position of the portion of the object is determined based at least in part on the intersection of the first line and the second line.

3. The unmanned aerial vehicle of claim 1 , wherein the at least one computer processor is further configured to at least:
select at least one of the second angular orientation or the second time based at least in part on the first predetermined speed.

4. The unmanned aerial vehicle of claim 1 , further comprising:
a second propulsion motor mounted to the frame, wherein the second propulsion motor is configured to rotate a second shaft about a second axis defined by the second shaft; and
a second propeller having a second plurality of blades, wherein the second propeller is rotatably coupled to the second shaft.

5. A method comprising:
capturing a first image of at least a portion of a scene by a first imaging device at a first time, wherein the first imaging device is integrated into a first surface of a first blade of a first propeller of a first aerial vehicle at a first radius from a first hub of the first propeller, and wherein the first propeller is rotating at a first angular velocity about a first axis of rotation at the first time;
capturing a second image of at least the portion of the scene by the first imaging device at a second time, wherein the first propeller is rotating at the first angular velocity about the first axis of rotation at the second time;
recognizing at least one point of the scene in the first image by at least one computer processor;
recognizing the at least one point of the scene in the second image by the at least one computer processor; and
determining a first range to the at least one point of the scene based at least in part on the first image and the second image by the at least one computer processor.

6. The method of claim 5 , further comprising:
determining a first position of the first imaging device at the first time;
determining a second position of the first imaging device at the second time; and
determining a baseline distance between the first position and the second position,
wherein the first range to the at least one point of the scene is determined based at least in part on the baseline distance.

7. The method of claim 6 , wherein the first blade is in a first angular orientation at the first time,
wherein the first blade is in a second angular orientation at the second time,
wherein the second angular orientation is approximately one hundred eighty degrees apart from the first angular orientation, and
wherein the baseline distance is approximately twice the first radius.

8. The method of claim 6 , wherein recognizing the at least one point of the scene in the first image comprises:
identifying a first representation of the at least one point of the scene in the first image by the at least one computer processor,
wherein recognizing the at least one point of the scene in the second image comprises:
identifying a second representation of the at least one point of the scene in the second image by the at least one computer processor, and
wherein determining the first range to the portion of the object comprises:
defining a first epipolar line from the first position through the first representation by the at least one computer processor;
defining a second epipolar line from the second position through the second representation by the at least one computer processor;
determining a position of an intersection of the first epipolar line and the second epipolar line by the at least one computer processor; and
determining the first range to the object based at least in part on the position of the intersection of the first epipolar line and the second epipolar line by the at least one computer processor.

9. The method of claim 5 , further comprising:
determining a first angular orientation of the first blade at the first time; and
determining a second angular orientation of the first blade at the second time,
wherein determining the first range to the at least one point of the scene based at least in part on the first image and the second image comprises:
aligning the first image and the second image with respect to one another based on a difference between the first angular orientation and the second angular orientation.

10. The method of claim 5 , wherein recognizing the at least one point of the scene in the first image comprises:
recognizing a plurality of points of the first image by the at least one computer processor,
wherein recognizing the at least one point of the scene in the second image comprises:
recognizing portions of at least some of the plurality of objects in the second image by the at least one computer processor, and
wherein determining the first range to the at least one point of the scene comprises:
determining ranges to the at least some of the plurality of points of the scene based at least in part on the first image and the second image by the at least one computer processor, wherein the first range is one of the ranges; and
defining a point cloud for the scene based at least in part on the ranges, wherein the depth map represents distances to each of a plurality of regions, and wherein each of the regions of the depth map corresponds to one of the portions of the at least some of the plurality of objects.

Assignees

• Amazon Tech Inc

Inventors

• Boyd Scott Patrick
• O'Brien Barry James
• Watson Joshua John
• Wilcox Scott Michael

Classifications

• B64U2201/00

  UAVs characterised by their flight controls · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• B64C11/00

  Propellers, e.g. of ducted type; Features common to propellers and rotors for rotorcraft · CPC title

• G03B35/02

  by sequential recording · CPC title