Method for robotic training based on randomization of surface damping

Patent metadata

Field	Value
Publication number	US-12017359-B2
Application number	US-202017095640-A
Country	US
Kind code	B2
Filing date	Nov 11, 2020
Priority date	Nov 11, 2020
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

A method, system and computer product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. Surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value are simulated. The Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

First claim

What is claimed is:

1. A method for training a control input system comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold;
c) simulating surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value; and
d) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

2. The method of claim 1 wherein simulating surface damping interactions includes simulating a time derivative penetration depth of the simulated environment.

3. The method of claim 2 wherein the penetration depth is randomized.

4. The method of claim 1 wherein a surface damping value of the simulated environment is randomized.

5. The method of claim 1 further comprising repeating steps a) through d).

6. The method of claim 5 wherein a surface damping value is randomized for each repetition.

7. The method of claim 1 wherein simulating surface damping interactions includes simulating dry friction forces.

8. The method of claim 1 wherein simulating surface damping interactions includes surface damping values of at least the simulated environment modeled as areas on a surface where each area has an associated surface damping value.

9. The method of claim 8 wherein the surface damping value of each area is randomly varied.

10. The method of claim 8 wherein the surface damping value of each of the areas is not constant and is generated using coherent noise.

11. The method of claim 8 wherein the surface damping value of each of the areas is constant and is generated using Gaussian noise or uniformly distributed noise.

12. The method of claim 8 wherein the surface damping value of a first subset of the areas is not constant and is generated using coherent noise and wherein the surface damping value of a second subset of the areas is constant and is generated using Gaussian noise or uniformly distributed noise.

13. The method of claim 8 wherein a shape of the areas is randomized.

14. The method of claim 8 wherein shapes of the areas are generated using coherent noise having at least one area shape based on a real object with noise added to the shape of the area.

15. The method of claim 8 wherein a shape of the areas is defined using coherent noise including the number of octaves, lacunarity, time persistence of the coherent noise or frequency distribution of the coherent noise.

16. The method of claim 1 wherein values of the surface damping of at least the simulated environment are modeled as a simplex or Perlin distribution of values on a surface.

17. A input control system comprising:
a processor;
a memory coupled to the processor;
non-transitory instruction embedded in the memory that when executed by the processor cause the processor to carry out the method for training control input comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more simulated movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes a simulated sensor value and a previous joint position if the integrated output value does not at least meet the threshold;
c) simulating surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more simulated movable joints based on an integral of the subsequent output value; and
d) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

18. The system of claim 17 wherein simulating surface damping interactions includes simulating a time derivative penetration depth of the simulated environment.

19. The system of claim 18 wherein the penetration depth is randomized.

20. The system of claim 17 wherein a surface damping value of the simulated environment is randomized.

21. The system of claim 17 further comprising repeating steps a) through d) wherein a surface damping value is randomized for each repetition.

22. A computer readable medium having non-transitory instruction embedded thereon that when executed cause a computer to carry out the method for training a control input system comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold;
c) simulating surface damping interactions with at least a simulated environment, a rigid body position and a position of the one or more movable joints based on an integral of the subsequent output value; and
d) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the simulated environment and position of the one or more movable joints.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Bashkirov Sergey
• Taylor Michael

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/048

  Activation functions · CPC title