Systems and methods autonomously performing instructed operations using a robotic device

Patent metadata

Field	Value
Publication number	US-11123871-B2
Application number	US-201916395418-A
Country	US
Kind code	B2
Filing date	Apr 26, 2019
Priority date	Apr 26, 2018
Publication date	Sep 21, 2021
Grant date	Sep 21, 2021

Abstract

Described in detail herein is autonomous system for performing instructed operations. A switch device in electrical communication with an autonomous robot device and a remote computing system, can replicate a first display of the autonomous robot device on a second display of the remote computing system. The switch device provides an interface for remotely controlling the autonomous robot device via the remote computing system. The autonomous robot device can control the articulated arm to move the at least one card to physically interact with a card reader. The remote computing system can autonomously extract each action performed by the autonomous robot device from the first display in response to replication of the first display on the second display via the switch device.

First claim

We claim:

1. An autonomous system for performing instructed operations, the system comprising:
a point-of-sale terminal including a processor and a card reader in electrical communication with the processor;
an autonomous robot device, the autonomous robot device includes a controller, an articulated arm, an image capture device, and a first display;
a remote computing system having a second display;
a switch device in electrical communication with the autonomous robot device and the remote computing system, the switch device being configured to replicate the first display on the second display and provide an interface for controlling the autonomous robot device;
wherein the autonomous robot device is configured to (i) receive a first set of instructions from the remote computing system via the interface of the switch device; (ii) pick up the at least one card from a plurality of cards using the articulated arm in response to the first set of instructions, each of the plurality of cards including at least one of a magnetic stripe or a microchip, and (ii) control the articulated arm to move the at least one card to physically interact with the card reader to facilitate reading of the magnetic stripe or microchip by the card reader in response to the first set of instructions;
wherein the remote computing system is configured to autonomously extract each action performed by the autonomous robot device from the first display in response to replication of the first display on the second display, and autonomously transmit a second set of instructions to the autonomous robot device via the interface of the switch device in response to detection of specified data in the first display that is replicated on the second display after the autonomous robot performs each action associated with the first set of instructions.

2. The system in claim 1 , wherein the switch device is a Keyboard Video Mouse (KVM) switch device.

3. The system of claim 1 , wherein the autonomous robot device is configured to interact with the card reader by swiping the magnetic stripe of the at least one card at the card reader or inserting the microchip of the at least one card into the card reader.

4. The system of claim 3 , wherein the card reader is configured to authenticate the at least one card in response to extracting and reading data encoded in the magnetic stripe or the microchip of the at least one card.

5. The system of claim 1 , wherein the point-of-sale terminal includes a third display, the third display configured to display information associated with the first set of instructions, in response to the autonomous robot device interfacing with the card reader.

6. The system of claim 5 , wherein the autonomous robot device is configured to:
stream, via the image capturing device of the autonomous robot device, one or more images including the information displayed on the third display;
display, via the first display of the autonomous robot device, the one or more images including the information associated with the first set of instructions displayed on the third display.

7. The system of claim 6 , wherein the second display of the remote computing system is configured to render the one or more images including the information associated with the first set of instructions.

8. The system of claim 7 , wherein the remote computing system is configured to:
extract the information associated with the first set of instructions from the image.

9. The system of claim 1 , wherein the card reader includes a data input device.

10. The system of claim 9 , wherein at least one action performed by the autonomous robot device includes inputting data using the data input device of the card reader.

11. A method for autonomously performing instructed operations, the method comprising:
replicating, via a switch device in electrical communication with an autonomous robot device and a remote computing system, a first display of the autonomous robot device on a second display of the remote computing system, the further including a controller, an articulated arm, and an image capture device;
providing, via the switch device, an interface for remotely controlling the autonomous robot device via the remote computing system;
receiving, by the autonomous robot device, a first set of instructions from the remote computing system via the interface of the switch device;
picking up, by the autonomous robot device, at least one card from a plurality of cards with the articulated arm in response to the first set of instructions, each of the plurality of cards including at least one of a magnetic stripe or a microchip;
controlling, via the autonomous robot device, the articulated arm to move the at least one card to physically interact with a card reader of a point of sale terminal including a processor in communication with the card reader, to facilitate reading of the magnetic stripe or microchip by the card reader in response to the first set of instructions;
autonomously extracting, via the remote computing system, each action performed by the autonomous robot device from the first display in response to replication of the first display on the second display via the switch device; and
transmitting, by the remote computing system, a second set of instructions to the autonomous robot device via the interface of the switch device in response to detection of specified data in the first display that is replicated on the second display after the autonomous robot performs each action associated with the first set of instructions.

12. The method in claim 11 , wherein the switch device is a Keyboard Video Mouse (KVM) switch device.

13. The method of claim 11 , further comprising, interacting with the card reader by swiping the magnetic stripe of the at least one card at the card reader or inserting the microchip of the at least one card in the card reader via the articulated arm of the autonomous robot device.

14. The method of claim 13 , further comprising, authenticating, via the card reader, the at least one card in response from extracting and reading data encoded in the magnetic stripe or the microchip of the at least one card.

15. The method of claim 11 , further comprising displaying, via a third display coupled to the point of sale terminal, information associated with the first set of instructions, in response to the at least one autonomous robot device interacting with the card reader.

16. The method of claim 15 , further comprising:
streaming, via the image capturing device of the autonomous robot device, one or more images of the information displayed on the third display;
displaying, via the first display of the autonomous robot device, the one or more images of the information associated with the first set of instructions displayed on the third display.

17. The method of claim 16 , further comprising rendering, via the second display of the remote computing system, the one or more images of the information associated with the first set of instructions.

18. The method of claim 17 , further comprising:
extracting, via the remote computing system, the information associated with the first set of instructions from the image.

19. The method of claim 11 , wherein the card reader includes a data input device.

20. The method of claim 19 , wherein at least one action includes inputting data using the data input device of the card reader.

Assignees

• Walmart Apollo Llc

Inventors

• Johnson Christopher D
• Fenton Timothy Mark

Classifications

• B25J9/1679 Primary

  characterised by the tasks executed · CPC title

• G05B2219/36115

  Card · CPC title

• G05B2219/50391

  Robot · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• G06Q20/204

  comprising interface for record bearing medium or carrier for electronic funds transfer or payment credit · CPC title