User feedback for robotic demonstration learning

Patent metadata

Field	Value
Publication number	US-11524402-B2
Application number	US-202016880866-A
Country	US
Kind code	B2
Filing date	May 21, 2020
Priority date	May 21, 2020
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for providing user feedback for robotic demonstration learning. One of the methods includes initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot. Local demonstration data is repeatedly collected for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template. A respective progress value for each of the one or more demonstration subtasks defined by the skill template is maintained. A user interface presentation is generated that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

First claim

What is claimed is:

1. A method performed by one or more computers, the method comprising:
initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot for use in training a control policy to control the robot to execute the skill template;
repeatedly collecting local demonstration data for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template;
maintaining a respective progress value for each of the one or more demonstration subtasks defined by the skill template that represents how much respective local demonstration data has been generated by the user manipulating the robot to perform the demonstration subtask for use in training the control policy; and
generating user interface presentation that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

2. The method of claim 1 , wherein the user interface presentation presents the progress value for a particular demonstration subtask relative to a target completion value for the particular demonstration subtask.

3. The method of claim 1 , wherein each respective progress value represents a number of iterations that the demonstration subtask has been demonstrated by the user physically manipulating the robot.

4. The method of claim 3 , wherein generating the user interface presentation comprises generating, for a particular demonstration subtask, information representing a suggested number of additional iterations for the particular demonstration subtask.

5. The method of claim 1 , wherein generating the user interface presentation comprises generating a presentation that shows respective progress of each demonstration subtask relative to each other demonstration subtask.

6. The method of claim 1 , further comprising:
receiving, through an initial user interface presentation, configuration parameters for a task to be performed by the robot according to the skill template.

7. The method of claim 1 , wherein generating the user interface presentation comprises generating a subtask animation that illustrates how to move the robot to provide local demonstration data for a particular subtask.

8. The method of claim 7 , wherein the particular subtask is a subtask selected by a user through a graphical user interface.

9. The method of claim 7 , wherein the particular subtask is a recommended next subtask.

10. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot for use in training a control policy to control the robot to execute the skill template;
repeatedly collecting local demonstration data for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template;
maintaining a respective progress value for each of the one or more demonstration subtasks defined by the skill template that represents how much respective local demonstration data has been generated by the user manipulating the robot to perform the demonstration subtask for use in training the control policy; and
generating user interface presentation that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

11. The system of claim 10 , wherein the user interface presentation presents the progress value for a particular demonstration subtask relative to a target completion value for the particular demonstration subtask.

12. The system of claim 10 , wherein each respective progress value represents a number of iterations that the demonstration subtask has been demonstrated by the user physically manipulating the robot.

13. The system of claim 12 , wherein generating the user interface presentation comprises generating, for a particular demonstration subtask, information representing a suggested number of additional iterations for the particular demonstration subtask.

14. The system of claim 10 , wherein generating the user interface presentation comprises generating a presentation that shows respective progress of each demonstration subtask relative to each other demonstration subtask.

15. The system of claim 10 , wherein the operations further comprise:
receiving, through an initial user interface presentation, configuration parameters for a task to be performed by the robot according to the skill template.

16. The system of claim 10 , wherein generating the user interface presentation comprises generating a subtask animation that illustrates how to move the robot to provide local demonstration data for a particular subtask.

17. The system of claim 16 , wherein the particular subtask is a subtask selected by a user through a graphical user interface.

18. The system of claim 17 , wherein the particular subtask is a recommended next subtask.

19. One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
initiating a local demonstration learning process to collect respective local demonstration data for each of one or more demonstration subtasks defined by a skill template to be executed by a robot for use in training a control policy to control the robot to execute the skill template;
repeatedly collecting local demonstration data for each of the one or more demonstration subtasks of the skill template while a user manipulates a robot to perform each of the one or more demonstration subtasks defined by the skill template;
maintaining a respective progress value for each of the one or more demonstration subtasks defined by the skill template that represents how much respective local demonstration data has been generated by the user manipulating the robot to perform the demonstration subtask for use in training the control policy; and
generating user interface presentation that presents a suggested demonstration to be performed by the user based on a respective progress value for each demonstration subtask.

20. The one or more non-transitory computer storage media of claim 19 , wherein the user interface presentation presents the progress value for a particular demonstration subtask relative to a target completion value for the particular demonstration subtask.

Assignees

• Intrinsic Innovation Llc

Inventors

• Kolluri Bala Venkata Sai Ravi Krishna

Classifications

• G05B2219/36184

  Record actions of human expert, teach by showing · CPC title

• B25J9/1661

  characterised by task planning, object-oriented languages · CPC title

• G05B2219/40391

  Human to robot skill transfer · CPC title

• G05B2219/39244

  Generic motion control operations, primitive skills each for special task · CPC title

• G05B2219/33056

  Reinforcement learning, agent acts, receives reward, emotion, action selective · CPC title