Distributed robotic demonstration learning

Patent metadata

Field	Value
Publication number	US-11679497-B2
Application number	US-202016880724-A
Country	US
Kind code	B2
Filing date	May 21, 2020
Priority date	May 21, 2020
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributed robotic demonstration learning. One of the methods includes receiving a skill template to be trained to cause a robot to perform a particular skill having a plurality of subtasks. One or more demonstration subtasks defined by the skill template are identified, wherein each demonstration subtask is an action to be refined using local demonstration data. On online execution system uploads sets of local demonstration data to a cloud-based training system. The cloud-based training system generates respective trained model parameters for each set of local demonstration data. The skill template is executed on the robot using the trained model parameters generated by the cloud-based training system.

First claim

What is claimed is:

1. A method comprising:
receiving, from a cloud-based training system by an online execution system configured to control a robot, a skill template to be trained to cause the robot to perform a particular skill having a plurality of subtasks;
identifying, by the online execution system, one or more demonstration subtasks defined by the skill template, wherein each demonstration subtask is an action to be refined using local demonstration data;
generating, by the online execution system, respective sets of local demonstration data for each of the one or more demonstration subtasks;
uploading, by the online execution system to the cloud-based training system, the sets of local demonstration data;
generating, by the cloud-based training system, respective trained model parameters for each set of local demonstration data, wherein the cloud-based training system comprises more computational capability than the online execution system;
receiving, by the online execution system, from the cloud-based training system, the trained model parameters generated by the cloud-based training system; and
executing, by the online execution system, the skill template using the trained model parameters generated by the cloud-based training system.

2. The method of claim 1 , wherein generating the respective sets of local demonstration data comprises generating task state representations for each of a plurality of points in time.

3. The method of claim 2 , wherein the task state representations each represent an output generated by a respective sensor that observes the robot.

4. The method of claim 1 , wherein the online execution system and the robot are located in a same facility, and wherein the cloud-based training system is accessible only over the Internet.

5. The method of claim 1 , further comprising:
receiving, by the online execution system from the cloud-based training system, a base control policy for each of the one or more demonstration subtasks.

6. The method of claim 5 , wherein executing the skill template comprises generating, by the online execution system, a corrective action using the trained model parameters generated by the cloud-based training system.

7. The method of claim 6 , further comprising adding the corrective action to a base action generated by the based control policy received from the cloud-based training system.

8. The system of claim 5 , wherein the base control policy is generated from system demonstration data generated on one or more different second robots at one or more facilities remote from a first robot.

9. The system of claim 1 , wherein the skill template can be adapted for multiple different robot types.

10. The system of claim 1 , wherein the cloud-based training system is a distributed system comprising one or more nodes.

11. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, from a cloud-based training system by an online execution system configured to control a robot, a skill template to be trained to cause the robot to perform a particular skill having a plurality of subtasks;
identifying, by the online execution system, one or more demonstration subtasks defined by the skill template, wherein each demonstration subtask is an action to be refined using local demonstration data;
generating, by the online execution system, respective sets of local demonstration data for each of the one or more demonstration subtasks;
uploading, by the online execution system to the cloud-based training system, the sets of local demonstration data;
generating, by the cloud-based training system, respective trained model parameters for each set of local demonstration data, wherein the cloud-based training system comprises more computational capability than the online execution system;
receiving, by the online execution system, from the cloud-based training system, the trained model parameters generated by the cloud-based training system; and
executing, by the online execution system, the skill template using the trained model parameters generated by the cloud-based training system.

12. The system of claim 11 , wherein generating the respective sets of local demonstration data comprises generating task state representations for each of a plurality of points in time.

13. The system of claim 12 , wherein the task state representations each represent an output generated by a respective sensor that observes the robot.

14. The system of claim 11 , wherein the online execution system and the robot are located in a same facility, and wherein the cloud-based training system is accessible only over the Internet.

15. The system of claim 11 , wherein the operations further comprise:
receiving, by the online execution system from the cloud-based training system, a base control policy for each of the one or more demonstration subtasks.

16. The system of claim 15 , wherein executing the skill template comprises generating, by the online execution system, a corrective action using the trained model parameters generated by the cloud-based training system.

17. The system of claim 16 , further comprising adding the corrective action to a base action generated by the based control policy received from the cloud-based training system.

18. One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving, from a cloud-based training system by an online execution system configured to control a robot, a skill template to be trained to cause the robot to perform a particular skill having a plurality of subtasks;
identifying, by the online execution system, one or more demonstration subtasks defined by the skill template, wherein each demonstration subtask is an action to be refined using local demonstration data;
generating, by the online execution system, respective sets of local demonstration data for each of the one or more demonstration subtasks;
uploading, by the online execution system to the cloud-based training system, the sets of local demonstration data;
generating, by the cloud-based training system, respective trained model parameters for each set of local demonstration data, wherein the cloud-based training system comprises more computational capability than the online execution system;
receiving, by the online execution system, from the cloud-based training system, the trained model parameters generated by the cloud-based training system; and
executing, by the online execution system, the skill template using the trained model parameters generated by the cloud-based training system.

19. The one or more non-transitory computer storage media of claim 18 , wherein generating the respective sets of local demonstration data comprises generating task state representations for each of a plurality of points in time.

20. The one or more non-transitory computer storage media of claim 19 , wherein the task state representations each represent an output generated by a respective sensor that observes the robot.

21. The one or more non-transitory computer storage media of claim 18 , wherein the online execution system and the robot are located in a same facility, and wherein the cloud-based training system is accessible only over the Inte

Assignees

• Intrinsic Innovation Llc

Inventors

• Kolluri Bala Venkata Sai Ravi Krishna
• Schaal Stefan
• Davis Benjamin M
• Schönherr Ralf Oliver Michael
• Ye Ning

Classifications

• B25J9/1656

  characterised by programming, planning systems for manipulators · CPC title

• B25J9/1689

  Teleoperation · CPC title

• B25J9/163 Primary

  learning, adaptive, model based, rule based expert control · CPC title

• B25J9/161 Primary

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• G05B2219/36442

  Automatically teaching, teach by showing · CPC title