Learning skills from video demonstrations

Patent metadata

Field	Value
Publication number	US-11524401-B1
Application number	US-202016829277-A
Country	US
Kind code	B1
Filing date	Mar 25, 2020
Priority date	Mar 28, 2019
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

A method includes determining motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task, determining a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information, determining an action to be performed by the system based on the motion imitation information and the predicted correction; and controlling motion of the system in accordance with the action.

First claim

What is claimed is:

1. A method, comprising:
determining motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task;
determining a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information;
determining an action to be performed by the system based on the motion imitation information and the predicted correction; and
controlling motion of the system in accordance with the action.

2. The method of claim 1 , wherein the motion information is based on a video that shows a demonstration of the physical task by an actor.

3. The method of claim 2 , wherein the motion information describes positions of parts of the actor at each of multiple time steps.

4. The method of claim 1 , wherein determining the predicted correction based on the motion information and the current state from the system comprises determining an encoded representation based on the motion information.

5. The method of claim 4 , wherein the encoded representation is provided to the second machine learning model as an input.

6. The method of claim 1 , wherein determining the predicted correction based on the motion information and the current state from the system comprises determining a tracking error that represents a difference between the current state and the motion information, wherein the tracking error is provided to the second machine learning model as an input.

7. The method of claim 1 , wherein the system is a simulated system and an error value for the simulated system is used as a supervision signal for reinforcement learning.

8. A non-transitory computer-readable storage device including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations, the operations comprising:
determining motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task;
determining a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information;
determining an action to be performed by the system based on the motion imitation information and the predicted correction; and
controlling motion of the system in accordance with the action.

9. The non-transitory computer-readable storage device of claim 8 , wherein the motion information is based on a video that shows a demonstration of the physical task by an actor.

10. The non-transitory computer-readable storage device of claim 9 , wherein the motion information describes positions of parts of the actor at each of multiple time steps.

11. The non-transitory computer-readable storage device of claim 8 , wherein determining the predicted correction based on the motion information and the current state from the system comprises determining an encoded representation based on the motion information.

12. The non-transitory computer-readable storage device of claim 8 , wherein determining the predicted correction based on the motion information and the current state from the system comprises determining a tracking error that represents a difference between the current state and the motion information, wherein the tracking error is provided to the second machine learning model as an input.

13. The non-transitory computer-readable storage device of claim 8 , wherein the system is a simulated system and an error value for the simulated system is used as a supervision signal for reinforcement learning.

14. An apparatus, comprising:
a memory; and
one or more processors that are configured to execute instructions that are stored in the memory, wherein the instructions, when executed, cause the one or more processors to:
determine motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task;
determine a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information;
determine an action to be performed by the system based on the motion imitation information and the predicted correction; and
control motion of the system in accordance with the action.

15. The apparatus of claim 14 , wherein the motion information is based on a video that shows a demonstration of the physical task by an actor.

16. The apparatus of claim 15 , wherein the motion information describes positions of parts of the actor at each of multiple time steps.

17. The apparatus of claim 14 , wherein the instructions that cause the one or more processors to determine the predicted correction based on the motion information and the current state from the system further cause the one or more processors to determine an encoded representation based on the motion information.

18. The apparatus of claim 14 , wherein the instructions that cause the one or more processors to determine the predicted correction based on the motion information and the current state from the system further cause the one or more processors to determine a tracking error that represents a difference between the current state and the motion information, wherein the tracking error is provided to the second machine learning model as an input.

19. The apparatus of claim 14 , wherein the system is a simulated system and an error value for the simulated system is used as a supervision signal for reinforcement learning.

20. The apparatus of claim 14 , wherein the system is a robotic system.

Assignees

• Zhang Jian
• Srouji Mario J
• Salakhutdinov Ruslan
• Apple Inc

Inventors

• Zhang Jian
• Srouji Mario J
• Salakhutdinov Ruslan

Classifications

• G05B2219/40391

  Human to robot skill transfer · CPC title

• G05B19/42

  Recording and playback systems, i.e. in which the program is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine · CPC title

• B25J9/1664 Primary

  characterised by motion, path, trajectory planning · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title