Reinforcement learning for human robot interaction

Patent metadata

Field	Value
Publication number	US-11494641-B2
Application number	US-201716651282-A
Country	US
Kind code	B2
Filing date	Dec 27, 2017
Priority date	Dec 27, 2017
Publication date	Nov 8, 2022
Grant date	Nov 8, 2022

Abstract

A system and method of teaching a neural network through reinforcement learning methodology. The system includes a machine-readable medium having one or more processors that perform a motion task to produce a first result corresponding to navigating a device during a first episode and performing an interaction task during that same episode. After completion of the first episode a processor calculates a Q value change based on the first task result and the second task result. The processor then modifies parameters based on the Q value change such that during subsequent episode iterations the motion task and interactive task are improved and a smooth and continuous transition occurs between these two tasks.

First claim

What is claimed is:

1. A system for teaching an artificial neural network using reinforcement learning, the system comprising:
a computer readable medium including instructions; and
one or more processors that, when the instructions are executed, is configured to:
perform a motion task to produce a first result, the motion task corresponding to navigating a device during a first episode, the motion task performed by a first multi-layer perceptron (MLP);
perform an interaction task to produce a second result during navigation of the first episode;
compute a Q value change from the first task result and the second task result using a second MLP;
modify parameters of the first MLP and the second MLP based on the Q value change; and
iteratively perform the motion task, the interaction task, computing the Q value change, and modifying parameters of the first MLP and the second MLP on episodes of the navigation subsequent to the first episode.

2. The system of claim 1 wherein transition between the motion task and interaction task is continuous.

3. The system of claim 1 wherein the first MLP is coupled to an object detection subnet and a motion history subnet to form an obstacle avoidance network.

4. The system of claim 3 wherein the motion history subnet obtains motion history data from a replay memory of the device.

5. The system of claim 1 wherein the first MLP is coupled to an obstacle avoidance network and a motion direction subnet to form a navigation network.

6. The system of claim 1 wherein the second MLP is coupled to the first MLP and a task driven subnet.

7. The system of claim 6 wherein the task driven subnet obtains task-related data from an input to guide the device to perform the interaction task.

8. The system of claim 1 wherein the second MLP provides a first task result weight and a second task result weight when computing the Q value change.

9. A method for caching an artificial neural network using reinforcement learning comprising:
performing a motion task to produce a first result, the motion task corresponding to navigating a device during a first episode, the motion task performed by a first multi-layer perceptron (MLP);
performing an interaction task to produce a second result during navigation of the first episode;
computing a Q value change from the first task result and the second task result using a second MLP;
modifying parameters of the first MLP and the second MLP based on the Q value change; and
iteratively performing the motion task, the interaction task, computing the Q value change, and modifying parameters of the first MLP and the second MLP on episodes of the navigation subsequent to the first episode.

10. The method of claim 9 wherein transition between the motion task and interaction task is continuous.

11. The method of claim 9 wherein the first MLP is coupled to an object detection subnet and a motion history subnet to form an obstacle avoidance network.

12. The method of claim 11 wherein the motion history subnet obtains motion history data from a replay memory of the device.

13. The method of claim 9 wherein the first MLP is coupled to an obstacle avoidance network and a motion direction subnet to form a navigation network.

14. The method of claim 9 wherein the second MLP is coupled to the first MLP and a task driven subnet.

15. The method of claim 14 wherein the task driven subnet obtains task-related data from an input to guide the device to perform the interaction task.

16. The method of claim 9 wherein the second MLP provides a first task result weight and a second task result weight when computing the Q value change.

17. At least one non-transitory machine readable medium including instructions for teaching an artificial neural network using reinforcement learning, the instructions, when executed by a machine, cause the machine to perform operations comprising:
perform a motion task to produce a first result, the motion task corresponding to navigating a device during a first episode, the motion task performed by a first multi-layer perceptron (MLP);
perform an interaction task to produce a second result during navigation of the first episode;
compute a Q value change from the first task result and the second task result using a second MLP;
modify parameters of the first MLP and the second MLP based on the Q value change; and
iteratively perform the motion task, the interaction task, computing the Q value change, and modifying parameters of the first MLP and the second MLP on episodes of the navigation subsequent to the first episode.

18. The at least one machine readable medium of claim 17 wherein transition between the motion task and interaction task is continuous.

19. The at least one machine readable medium of claim 17 wherein the first MLP is coupled to an object detection subnet and a motion history subnet to form an obstacle avoidance network.

20. The at least one machine readable medium of claim 19 wherein the motion history subnet obtains motion history data from a replay memory of the device.

21. The at least one machine readable medium of claim 17 wherein the first MLP is coupled to an obstacle avoidance network and a motion direction subnet to form a navigation network.

22. The at least one machine readable medium of claim 17 wherein the second MLP is coupled to the first MLP and a task driven subnet.

23. The at least one machine readable medium of claim 22 wherein the task driven subnet obtains task-related data from an input to guide the device to perform the interaction task.

24. The at least one machine readable medium of claim 17 wherein the second MLP provides a first task result weight and a second task result weight when computing the Q value change.

Assignees

• Intel Coprporation
• Intel Corp

Inventors

• Chen Hu Tiger
• Liu Zhongxuan
• Zhang Yimin
• Ren Haibing
• Hu Jiankun

Classifications

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G05B15/02

  electric · CPC title