Selectively paired imaging elements for stereo images

Patent metadata

Field	Value
Publication number	US-10706569-B2
Application number	US-201615176734-A
Country	US
Kind code	B2
Filing date	Jun 8, 2016
Priority date	Jun 8, 2016
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

This disclosure describes a configuration of an aerial vehicle, such as an unmanned aerial vehicle (“UAV”), that includes a plurality of cameras that may be selectively combined to form a stereo pair for use in obtaining stereo images that provide depth information corresponding to objects represented in those images. Depending on the distance between an object and the aerial vehicle, different cameras may be selected for the stereo pair based on the baseline between those cameras and a distance between the object and the aerial vehicle. For example, cameras with a small baseline (close together) may be selected to generate stereo images and depth information for an object that is close to the aerial vehicle. In comparison, cameras with a large baseline may be selected to generate stereo images and depth information for an object that is farther away from the aerial vehicle.

First claim

What is claimed is:

1. An aerial vehicle apparatus, comprising:
a frame;
a plurality of cameras coupled to the frame;
a camera selection controller in communication with each of the plurality of cameras, the camera selection controller configured to at least:
during aerial navigation of the aerial vehicle apparatus:
detect an object;
determine a first approximate distance between the aerial vehicle apparatus and the object;
select a first camera and a second camera from the plurality of cameras as a first combination of cameras, wherein the selection is based at least in part on the first approximate distance and a first baseline distance between the first camera and the second camera;
receive a first image from the first camera;
receive a second image from the second camera;
process the first image and the second image to generate first depth information corresponding to the object;
subsequent to generation of the first depth information, determine a second approximate distance between the aerial vehicle apparatus and the object;
determine that the second approximate distance is different than the first approximate distance;
in response to a determination that the second approximate distance is different than the first approximate distance, select a third camera and a fourth camera of the plurality of cameras as a second combination of cameras, wherein the selection is based at least in part on the second approximate distance and a second baseline distance between the second camera and the third camera;
receive a third image from the third camera;
receive a fourth image from the fourth camera; and
process the third image and the fourth image to generate second depth information corresponding to the object.

2. The aerial vehicle apparatus of claim 1 , wherein the camera selection controller is further configured to at least determine the second approximate distance based at least in part on the first depth information.

3. The aerial vehicle apparatus of claim 1 , further comprising:
a distance determining element coupled to the frame of the aerial vehicle apparatus and configured to at least determine at least one of the first approximate distance or the second approximate distance.

4. The aerial vehicle apparatus of claim 1 , wherein a first alignment of the first baseline distance is different than a second alignment of the second baseline distance.

5. The aerial vehicle apparatus of claim 1 , wherein the camera selection controller is further configured to at least:
select the first camera of the plurality of cameras and the fourth camera of the plurality of cameras as a third combination of cameras;
receive a fifth image from the first camera;
receive a sixth image from the fourth camera; and
process the fifth image and the sixth image to generate third depth information, wherein the third depth information includes information corresponding to the object.

6. The aerial vehicle apparatus of claim 1 , wherein the plurality of cameras coupled to the frame are spaced such that each potential pair of cameras of the plurality of cameras have a different baseline distance.

7. An aerial vehicle apparatus, comprising:
a propulsion mechanism;
a plurality of imaging elements spaced about the aerial vehicle apparatus;
an imaging element selection controller in communication with each of the plurality of imaging elements, the imaging element selection controller configured to at least:
during aerial navigation of the aerial vehicle apparatus:
detect an object;
determine a first approximate distance between the aerial vehicle apparatus and the object;
select a first combination of imaging elements of the plurality of imaging elements, wherein the selection is based at least in part on the first approximate distance and a first baseline distance between the imaging elements of the first combination of imaging elements;
obtain, from each imaging element of the first combination of imaging elements, a first image;
process the first image received from each imaging element of the first combination of imaging elements to generate first depth information corresponding to the object;
subsequent to generation of the first depth information, determine a second approximate distance between the aerial vehicle apparatus and the object;
determine that the second approximate distance is different than the first approximate distance;
in response to a determination that the second approximate distance is different than the first approximate distance, select a second combination of imaging elements of the plurality of imaging elements, wherein the selection is based at least in part on the second approximate distance and a second baseline distance between the imaging elements of the second combination of imaging elements;
obtain, from each imaging element of the second combination of imaging elements, a second image; and
process the second image received from each imaging element of the second combination of imaging elements to generate second depth information corresponding to the object.

8. The aerial vehicle apparatus of claim 7 , wherein at least three of the plurality of imaging elements are spaced about the aerial vehicle apparatus such that a baseline distance between each of the at least three imaging elements of the plurality of imaging elements is different.

9. The aerial vehicle apparatus of claim 7 , wherein:
the first approximate distance is greater than the second approximate distance; and
the first baseline distance is larger than the second baseline distance.

10. The aerial vehicle apparatus of claim 7 , wherein:
a first alignment of the first baseline distance between each imaging element of the first combination of imaging elements is different than a second alignment of the second baseline distance between each imaging element of the second combination of imaging elements.

11. The aerial vehicle apparatus of claim 7 , wherein a position of the object is within a common field of view of each imaging element of the first combination of imaging elements.

12. The aerial vehicle apparatus of claim 7 , wherein at least a portion of a field of view of each of the plurality of imaging elements overlap.

13. The aerial vehicle apparatus of claim 7 , wherein:
images are periodically generated by each of the plurality of imaging elements; and
the imaging element selection controller obtains a first image from a data store of the aerial vehicle apparatus in response to selecting a first imaging element as an imaging element of the first combination of imaging elements.

14. The aerial vehicle apparatus of claim 7 , wherein the imaging element selection controller is further configured to at least:
generate, based at least in part on the first image received from each imaging element of the first combination of imaging elements, first disparity information;
select a third combination of imaging elements, wherein the selection is based at least in part on the first approximate distance and a third baseline distance between the imaging elements of the third combination of imaging elements;
obtain, from each imaging element of the third combination of imaging elements, a third image;
generate, based at least in part on the third image received from each imaging element of the third combination of imaging elements, second disparity information; and
determine to process the first image received from each imaging element of the first combination of imaging elements to generate first depth information corresponding to the object based at least in part on a determination that a first disparity of t

Assignees

• Amazon Tech Inc

Inventors

• Harris Scott Raymond
• O'Brien Barry James
• Watson Joshua John

Classifications

• G06T7/596 Primary

  from three or more stereo images · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T7/55 Primary

  from multiple images · CPC title

• G06T2207/30181

  Earth observation · CPC title