Systems and methods for distributed autonomous robot interfacing using live image feeds

Patent metadata

Field	Value
Publication number	US-10494180-B2
Application number	US-201816217865-A
Country	US
Kind code	B2
Filing date	Dec 12, 2018
Priority date	Jan 30, 2017
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

Described in detail herein is an autonomous fulfillment system. The system includes the first computing system with an interactive display. The first computing system can transmit a request for physical objects from a facility. A second computing system can transmit instructions to autonomous robot devices to retrieve the physical objects from the facility. The second computing system can control the image capturing device of an autonomous robot device to capture a live image feed of at least one physical object picked up by the autonomous robot device. The second computing system can switch an input feed of the first computing system to display the live image feed on the interactive display of the first computing system. The second computing system can instruct the autonomous robot device to discard the physical objects picked up by the autonomous robot device and to pick up a replacement physical object.

First claim

We claim:

1. An autonomous system including autonomous robot devices, the system comprising:
a first computing system including an interactive display configured to transmit a first request for a plurality of physical objects from a facility;
a plurality of autonomous robot devices including a controller, a drive motor, an articulated arm, a reader, an inertial navigation system and an image capturing device, the plurality of autonomous robot devices configured to navigate autonomously through the facility;
a database in communication with the plurality of robots, configured to store information associated with the plurality of physical objects;
a second computing system in communication with the first computing system, the database, and the plurality of autonomous robot devices, the second computing system configured to:
receive the first request for the plurality of physical objects from the first computing system;
transmit instructions the at least one of the plurality of autonomous robot devices to retrieve the plurality of physical objects from the facility;
detect the at least one autonomous robot device has picked up at least one of the plurality of physical objects;
control the image capturing device of the at least autonomous robot device to capture a live image feed of the at least one physical object picked up by the at least autonomous robot device;
switch an input feed of the first computing system to display the live image feed on the display of the first computing system;
receive a second request from first computing system to discard the at least one of the physical objects picked up by the at least one autonomous robot device and to pick up a replacement physical object; and
transmit instructions to the at least one autonomous device to discard the at least one of the physical objects picked up by the at least one autonomous robot device and to pick up the replacement physical object, the instructions include one or more identifiers for the plurality of physical objects.

2. The system of claim 1 , wherein the at least one of the autonomous robot further devices configured to:
query the database using the one or more identifiers for the plurality of physical objects to retrieve the locations at which the plurality of physical objects are disposed;
navigate autonomously through the facility to the locations in response to operation of the drive motor by the controller;
locate and scan one or more machine readable elements encoded with the one or more identifiers;
detect, via at least one image captured by the image capture device, that the plurality of physical objects are disposed at the locations;
pick up a first quantity of each of the plurality of physical objects using the articulated arm.

3. The system in claim 2 , wherein the at least one of the autonomous robot further devices configured to:
carry and navigate with the first quantity of each of the plurality of physical objects to a specified location.

4. The system in claim 3 , further comprising a plurality of sensors disposed at the locations of each of the plurality of physical objects, the plurality of sensors configured to determine a set of attributes associated with each of the plurality of physical objects retrieved by the at least one autonomous robot device.

5. The system in claim 4 , wherein the second computing system is further configured to transmit an alert to be displayed on the display of the first computing system based on the detected set of attributes for the at least one of the plurality of physical objects retrieved by the at least one autonomous robot device.

6. The system in claim 3 , wherein the first computing system is disposed in the facility and the specified location is a location of the first computing system within the facility.

7. The system in claim 6 , wherein the at least one autonomous robot device is further configured to navigate to a storage container within the facility, deposit the first quantity of each of the plurality of physical object in the storage container, pick up the storage container and carry and navigate the storage container to the specified location.

8. The system in claim 7 , wherein the specified location is a parking lot in which a plurality of automobiles are disposed.

9. The system in claim 8 , wherein the first request received by the second computing system includes an identification number associated with at least one of the plurality of automobile and a tag number associated with the at least one automobile.

10. The system in claim 8 , wherein the at least one autonomous robotic device is further configured to:
carry the storage container to the parking lot;
detect using the image capturing device, the identification number of the at least one automobile;
navigate to the at least one automobile;
autonomously open an access point of the at least one automobile using the tag number; and
deposit the storage container inside the at least one automobile, via the access point.

11. An autonomous method including autonomous robot devices, the method comprising:
transmitting, via a first computing system including an interactive display, a first request for a plurality of physical objects from a facility;
navigating, via a plurality of autonomous robot devices including a controller, a drive motor, an articulated arm, a reader, an inertial navigation system and an image capturing device, autonomously through the facility;
receiving, via a second computing system in communication with the first computing system and the plurality of autonomous robot devices, the first request for the plurality of physical objects from the first computing system;
transmitting, via the second computing system, instructions the at least one of the plurality of autonomous robot devices to retrieve the plurality of physical objects from the facility;
detecting, via the second computing system, the at least one autonomous robot device has picked up at least one of the plurality of physical objects;
controlling, via the second computing system, the image capturing device of the at least autonomous robot device to capture a live image feed of the at least one physical object picked up by the at least autonomous robot device;
switching, via the second computing system, an input feed of the first computing system to display the live image feed on the display of the first computing system;
receiving, via the second computing system, a second request from first computing system to discard the at least one of the physical objects picked up by the at least one autonomous robot device and to pick up a replacement physical object;
transmitting, via the second computing system, instructions to the at least one autonomous device to discard the at least one of the physical objects picked up by the at least one autonomous robot device and to pick up the replacement physical object, the instructions from the second computing system include one or more identifiers for the plurality of physical objects.

12. The method of claim 11 , further comprising:
querying, via the at least one of the autonomous robot devices, a database operatively coupled to the second computing system and the plurality of the autonomous robot devices using the one or more identifiers for the plurality of physical objects;
navigating, via the at least one autonomous robot device, autonomously through the facility to the first set of object locations in response to operation of the drive motor by the controller;
locating and scanning, via the at least one autonomous robot device, one or more machine readable elements encoded with the one or more identifiers;
detecting, via at

Assignees

• Walmart Apollo Llc

Inventors

• High Donald R
• Winkle David
• Mchale Brian Gerard
• Mattingly Todd Davenport

Classifications

• Y10S901/01

  Mobile robot · CPC title

• G05B2219/40507

  Distributed planning, offline trajectory, online motion, avoid collision · CPC title

• G05B2219/36412

  Fine, autonomous movement of end effector by using camera · CPC title

• G05B2219/32388

  Autonomous flexible system, cells and agv autonomous · CPC title

• G05B2219/23363

  Barcode · CPC title