Control input scheme for machine learning in motion control and physics based animation

Patent metadata

Field	Value
Publication number	US-12236339-B2
Application number	US-201916693093-A
Country	US
Kind code	B2
Filing date	Nov 22, 2019
Priority date	Nov 22, 2019
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

A method, system and non-transitory instructions for control input, comprising, taking an integral of an output value from a Motion Decision Neural Network for a movable joint to generate an integrated output value. Generating a subsequent output value using a machine learning algorithm that includes a sensor value and the integrated output value as inputs to the Motion Decision Neural Network and imparting movement with the moveable joint according to an integral of the subsequent output value.

First claim

What is claimed is:

1. A method for control input, comprising:
a) taking an integral of an output value from a trained Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes each of a sensor value, the integrated output value, and one or more of visual information, sound information, motion information, as inputs to the trained Motion Decision Neural Network, wherein the sensor value is generated by one or more sensors and the one or more of visual information, sound information, motion information is generated by one or more other sensors, wherein the one or more sensors is a different type of sensor than the one or more other sensors; and
c) imparting movement with the one or more moveable joints according to an integral of the subsequent output value.

2. The method of claim 1 , wherein the movable joint is in a virtual simulation.

3. The method of claim 1 , wherein the movable joint is a motorized joint.

4. The method from claim 1 , wherein c) includes changing a position of the one or more movable joints using the subsequent output value and updating the sensor value and providing the subsequent integrated output value to the motion decision NN.

5. The method of claim 4 , further comprising repeating steps a) through c).

6. The method of claim 1 , wherein the integral of an output value is a first integral of the output value and the integral of the subsequent output value is a first integral of the subsequent output value.

7. The method of claim 1 , wherein the integral of an output value is a second integral of the output value and the integral of the subsequent output value is a second integral of the subsequent output value.

8. The method of claim 7 , further comprising taking a first integral of the output value and wherein the integrated output value includes the first integral of the output value and the second integral of the output value.

9. The method of claim 1 , wherein the visual information, sound information, or motion information indicate the presence of a person or other important object.

10. The method of claim 1 , wherein the sensor value corresponds to an output of a sensor on a robot.

11. The method of claim 10 , wherein the sensor value corresponds to one or more of a joint position, a joint velocity, a joint torque, a robot orientation, a robot linear velocity, a robot angular velocity, a foot contact point, a foot pressure or some combination of two or more of these.

12. The method of claim 1 , wherein the sensor value corresponds to an output of virtual sensor of a robot simulation.

13. The method of claim 12 , wherein the sensor value corresponds to a joint position, a joint velocity, a joint torque, a character orientation, a model linear velocity, a character angular velocity, a foot contact point, a foot pressure, or some combination of two or more of these.

14. The method of claim 1 , wherein the control input is a control input of a video game.

15. The method of claim 1 , wherein the control input is a control input of a cloud game.

16. The method of claim 1 , wherein the control input is a control input of a game development engine.

17. A system for motion control, comprising:
a processor;
a memory coupled to the processor;
non-transitory instructions embedded in the memory that when executed by the processor cause the processor to carry out the method comprising:
a) taking an integral of an output value from a trained Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes each of a sensor value, the integrated output value, and one or more of visual information, sound information, motion information, as inputs to the trained Motion Decision Neural Network, wherein the sensor value is generated by one or more sensors and the one or more of visual information, sound information, motion information is generated by one or more other sensors, wherein the one or more sensors is a different type of sensor than the one or more other sensors; and
c) imparting movement with the one or more moveable joints according to an integral of the subsequent output value.

18. The system of claim 17 , wherein the movable joint is in a virtual simulation.

19. The system of claim 17 , further comprising a motorized movable joint and wherein the movable joint is the motorized movable joint.

20. The system from claim 17 , wherein c) includes changing a position of the one or more movable joints using the subsequent output value and updating the sensor value and providing the subsequent integrated output value to the motion decision NN.

21. The system of claim 20 , further comprising repeating steps a) through c).

22. The system of claim 17 , wherein the integral of an output value is a first integral of the output value and the integral of the subsequent output value is a first integral of the subsequent output value.

23. The system of claim 17 , wherein the integral of an output value is a second integral of the output value and the integral of the subsequent output value is a second integral of the subsequent output value.

24. The system of claim 23 , further comprising taking a first integral of the output value and wherein the integrated output value includes the first integral of the output value and the second integral of the output value.

25. The system of claim 17 , wherein the visual information, sound information, or motion information indicate the presence of a person or other important object.

26. The system of claim 17 , wherein the sensor value corresponds to an output of a sensor on a robot.

27. The system of claim 26 , wherein the sensor value corresponds to one or more of a joint position, a joint velocity, a joint torque, a robot orientation, a robot linear velocity, a robot angular velocity, a foot contact point, a foot pressure or two or more of these.

28. The system of claim 17 , wherein the sensor value corresponds to a virtual sensor of a robot simulation.

29. The system of claim 28 , wherein the sensor value corresponds to a joint position, a joint velocity, a joint torque, a character orientation, a character linear velocity, a character angular velocity, a character foot contact point, a foot pressure or two or more of these.

30. The system of claim 17 , wherein the control input is a control input of a video game.

31. The system of claim 17 , wherein the control input is a control input of a cloud game.

32. The system of claim 17 , wherein the control input is a control input of a game development engine.

33. Non-transitory instructions embedded in a computer readable medium that when executed by a computer cause the computer to carry out the method comprising:
a) taking an integral of an output value from a trained Motion Decision Neural Network for one or more movable joints to generate an integrated output value;
b) generating a subsequent output value using a machine learning algorithm that includes each of a sensor value, the integrated output value, and one or more of visual information, sound information, motion information, as inputs to the t

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Taylor Michael
• Bashkirov Sergey

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title