Domain adaptation for simulated motor backlash

Patent metadata

Field	Value
Publication number	US-12280499-B2
Application number	US-202017095586-A
Country	US
Kind code	B2
Filing date	Nov 11, 2020
Priority date	Nov 11, 2020
Publication date	Apr 22, 2025
Grant date	Apr 22, 2025

Abstract

A method, system and computer program product for training a control input system involve taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value and comparing the integrated output value to a backlash threshold. A subsequent output value is generated using a machine learning algorithm that includes a sensor value and a previous joint position if the integrated output value does not at least meet the threshold. A position of the one or more movable joints is simulated based on an integral of the subsequent output value; and the Motion Decision Neural Network is trained with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints.

First claim

What is claimed is:

1. A method for training a control input system, comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) comparing the integrated output value to a backlash threshold;
c) generating a subsequent output value using a machine learning algorithm that includes as inputs a sensor value and a previous joint position when the integrated output value does not at least meet the threshold;
d) simulating a position of the one or more movable joints based on an integral of the subsequent output value; and
e) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints;
repeating a) through e), wherein c) includes generating the subsequent output value using a machine learning algorithm that includes a sensor value and the integrated output value when the integrated output value meets or exceeds the threshold; and
controlling a robot by passing the integrated output value to a movable joint when the integrated output value meets or exceeds the threshold.

2. The method of claim 1 , wherein the movable joint is in a virtual simulation.

3. The method of claim 1 , wherein the movable joint is a motorized joint.

4. The method of claim 1 , wherein the integral of the output value is a first integral of the output value and the integral of the subsequent output value is a first integral of the subsequent output value.

5. The method of claim 1 , wherein the integral of the output value is a second integral of the output value and the integral of the subsequent output value is a second integral of the subsequent output value.

6. The method of claim 5 , further comprising taking a first integral of the output value and wherein the integrated output value includes the first integral of the output value and the second integral of the output value.

7. The method of claim 1 , wherein the threshold is a randomized value.

8. The method of claim 7 , wherein steps a) through e) are repeated with a different randomized value each repetition.

9. The method of claim 1 , wherein the threshold is at least based on a time or a number of repetitions of steps a) through e).

10. The method of claim 9 , wherein the threshold value changes with a change in the time.

11. The method of claim 10 , wherein at least one of the one or more movable joints is associated with a threshold value that changes differently with the change in time than another movable joint of the one or more movable joints.

12. The method of claim 1 , wherein the threshold is different for at least one of the one or more movable joint compared to another joint of the one or more movable joints.

13. The method of claim 12 , wherein the different threshold depends at least upon a location of the at least one of the one or more movable joint in a simulation.

14. The method of claim 1 , wherein the threshold value at least depends upon an angle value of at least one of the one or more movable joints.

15. The method of claim 14 , wherein the threshold value changes based on the amount of times the at least one of the one or more joints passes through the angle value.

16. The method of claim 14 wherein the angle dependent threshold values are represented by randomized heat map to simulate wear in the at least one of the one or more different joints or transitions between surfaces.

17. The method of claim 16 wherein the heatmap simulates a transition of the joints from areas of high use to areas of lower use.

18. The method of claim 17 , wherein different angles of the joints have different threshold valued for different surface types.

19. A input control system comprising:
a processor;
a memory coupled to the processor;
non-transitory instruction embedded in the memory that when executed by the processor cause the processor to carry out the method for training control input comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) comparing the integrated output value to a backlash threshold;
c) generating a subsequent output value using a machine learning algorithm that includes as inputs a simulated sensor value and a previous simulated joint position when the integrated output value does not at least meet the threshold;
d) simulating a position of the one or more simulated movable joints based on an integral of the subsequent output value; and
e) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more simulated movable joints;
repeating a) through e), wherein c) includes generating the subsequent output value using a machine learning algorithm that includes a sensor value and the integrated output value when the integrated output value meets or exceeds the threshold; and
controlling a robot by passing the integrated output value to a movable joint when the integrated output value meets or exceeds the threshold.

20. A non-transitory computer readable medium having instruction embedded thereon that when executed cause a computer to carry out the method for training a control input system comprising:
a) taking an integral of an output value from a Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) comparing the integrated output value to a backlash threshold;
c) generating a subsequent output value using a machine learning algorithm that includes as inputs a sensor value and a previous joint position when the integrated output value does not at least meet the threshold;
d) simulating a position of the one or more movable joints based on an integral of the subsequent output value; and
e) training the Motion Decision Neural Network with the machine learning algorithm based upon at least a result of the simulation of the position of the one or more movable joints;
repeating a) through e), wherein c) includes generating the subsequent output value using a machine learning algorithm that includes a sensor value and the integrated output value when the integrated output value meets or exceeds the threshold; and
controlling a robot by passing the integrated output value to a movable joint when the integrated output value meets or exceeds the threshold.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Bashkirov Sergey
• Taylor Michael

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/048

  Activation functions · CPC title