Integrated obstacle detection and payload centering sensor system

Patent metadata

Field	Value
Publication number	US-10012992-B2
Application number	US-201615265925-A
Country	US
Kind code	B2
Filing date	Sep 15, 2016
Priority date	Sep 15, 2016
Publication date	Jul 3, 2018
Grant date	Jul 3, 2018

Abstract

Disclosed are various embodiments for an integrated obstacle detection and payload centering sensor system. A robotic drive unit (RDU) captures, with a downward facing camera mounted to itself, an image of fiducial located on the ground. The RDU then positions itself over the fiducial and subsequently rotates. As it rotates, the RDU captures a point cloud of the surround vicinity a forward-facing three-dimensional camera mounted to itself. The RDU then identifies in the point cloud at least two legs of a storage unit positioned over the robotic drive unit. Subsequently, the RDU determines a location for each of the at least two legs relative to the fiducial and triangulates a center of the storage unit based at least in part on the location of each of the at least two legs. The RDU then centers itself underneath the storage unit.

First claim

Therefore, the following is claimed:

1. A system, comprising:
a downward facing camera;
a forward facing camera;
a processor; and
a memory storing machine-readable instructions that, when executed by the processor, cause the system to at least:
move underneath a storage unit;
capture an image of a fiducial located on the ground using the downward facing camera;
center the system on the fiducial;
rotate the system in place around the fiducial;
capture a panoramic image with the forward facing camera as the system rotates;
identify a plurality of legs of the storage unit in the panoramic image;
triangulate a location of the center of the storage unit relative to the fiducial based at least in part on the plurality of legs in the panoramic image;
determine that the system is centered relative to the storage unit;
lift the storage unit;
capture a first image of at least two the plurality of the legs of the storage unit using the forward facing camera in response to causing the system to lift the storage unit;
move the storage unit to a new location;
capture a second image of the at least two of the plurality of legs of the storage unit using the forward facing camera while the system is in motion;
measure a change in position of the at least two of the plurality of legs of the storage unit in the second image compared to the first image; and
determine that the storage unit has changed position or orientation on top of the system based at least in part on the change in position.

2. The system of claim 1 , wherein the forward facing camera comprises a three-dimensional camera configured to capture a point cloud and the panoramic image comprises a point cloud.

3. The system of claim 1 , wherein the downward facing camera is located on the bottom surface of the system.

4. A robotic drive unit, comprising:
a forward facing camera;
a downward facing camera;
a processor;
a memory; and
an application comprising machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
move under a storage unit;
capture an image of a fiducial located on the ground using the downward facing camera;
position the robotic drive unit relative to the fiducial;
rotate the robotic drive unit in place;
capture a panoramic image with the forward facing camera;
identify a plurality of legs of the storage unit in the panoramic image;
measure a distance from each of the plurality of legs of the storage unit to the fiducial;
calculate a location under the storage unit relative to the fiducial;
determine that the robotic drive unit is out of position relative to the calculated location under the storage unit;
travel to the calculated location under the storage unit; and
lift the storage unit.

5. The robotic drive unit of claim 4 , further comprising a network interface and wherein:
the fiducial comprises a first fiducial; and
the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
identify in the panoramic image a second fiducial on one of the plurality of legs of the storage unit;
extract an identifier of the storage unit from the second fiducial; and
send a request via the network interface to a remote computing device, wherein the request comprises the identifier of the storage unit.

6. The robotic drive unit of claim 4 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
capture a first image of at least two of the plurality of the legs of the storage unit using the forward facing camera in response to causing the robotic drive unit to lift the storage unit;
move the storage unit to a new location;
capture a second image of the at least two of the plurality of legs of the storage unit using the forward facing camera while the robotic drive unit is in motion;
measure a change in position of the at least two of the plurality of legs of the storage unit in the second image compared to the first image; and
determine that the storage unit has changed position or orientation on top of the robotic drive unit based at least in part on the change in position.

7. The robotic drive unit of claim 4 , further comprising a network interface and wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
capture a first image using the forward facing camera in response to causing the robotic drive unit to lift the storage unit;
determine that at least two of the plurality of the legs of the storage unit are in the first image;
move the storage unit to a new location;
capture a second image using the forward facing camera while the robotic drive unit is in motion;
determine that the at least two of the plurality of legs of the storage unit are missing from the second image; and
send an alert via the network interface to a remote computing system, wherein the alert indicates that the storage unit has fallen from the robotic drive unit.

8. The robotic drive unit of claim 4 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
capture a first image of at least two of the plurality of the legs of the storage unit using the forward facing camera while the robotic drive unit is in motion;
capture a second image of the at least two of the plurality of legs of the storage unit using the forward facing camera while the robotic drive unit is in motion; and
measure a vertical change in position of the at least two of the plurality of legs of the storage unit in the second image compared to the first image.

9. The robotic drive unit of claim 8 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to calculate a likelihood that the storage unit will tip over based at least in part on the vertical change in position of the at least two of the plurality of legs of the storage unit.

10. The robotic drive unit of claim 8 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to calculate an amount of sway of the storage unit while the robotic drive unit is in motion.

11. The robotic drive unit of claim 8 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to calculate an amount of vertical displacement of the storage unit while robotic drive unit is in motion.

12. The robotic drive unit of claim 4 , wherein the downward facing camera is located on the bottom surface of the robotic drive unit.

13. The robotic drive unit of claim 4 , wherein the forward facing camera comprises a three-dimensional camera configured to capture a point cloud.

14. The robotic drive unit of claim 13 , wherein the panoramic image comprises the point cloud.

15. The robotic drive unit of claim 4 , wherein the application further comprises machine readable instructions stored in the memory that, when executed by the processor, cause the robotic drive unit to:
move the storage unit;
capture an image using the forward facing camera; and
identify an obstacle in the image that blocks a current path of the robotic drive unit.

Assignees

• Amazon Tech Inc

Inventors

• Shydo Jr Robert Michael

Classifications

• Y10S901/01

  Mobile robot · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• G05B19/4061

  Avoiding collision or forbidden zones · CPC title

• G05B2219/39046

  Compare image of plate on robot with reference, move till coincidence, camera · CPC title

• G05D2201/0216

  Physics · mapped topic