Method for improving operation of a robot

Patent metadata

Field	Value
Publication number	US-2016288323-A1
Application number	US-201615084705-A
Country	US
Kind code	A1
Filing date	Mar 30, 2016
Priority date	Apr 2, 2015
Publication date	Oct 6, 2016
Grant date	—

Abstract

The invention relates to a method for improving operation of at least one robot. The robot is being operated on the basis of a set of predefined actions. A method comprises generating combined actions by combining at least two actions out of a set of original actions stored in an action library. Storing the combined actions in the actions library in addition to the original actions. Applying a reinforcement learning algorithm to the set of actions stored now in the action library to learn a control policy making use of the original actions and the combined actions. And finally, operating the robot on the basis of the resulting action library.

First claim

1 . A method for improving operation of at least one robot being operated on the basis of a set of predefined actions, the method comprising:
generating combined actions by combining at least two actions out of a set of original actions stored in an action library, storing the combined actions in the action library in addition to the original actions, applying a reinforcement learning algorithm to the set of actions stored now in the action library to learn a control policy making use of the original actions and the combined actions, and operating the robot on the basis of the resulting action library.

2 . The method according to claim 1 , wherein
combined actions that are determined not to be used after the reinforcement learning step are removed from the action library.

3 . The method according to claim 1 , wherein
that the combined actions are combinations of two original actions wherein such combination is performed for all possible pairs of original actions out of all original actions of the action library.

4 . The method according to claim 3 , wherein
determining which of the combined actions are impossible and omitting storing those impossible combined actions in the library.

5 . The method according to claim 1 , wherein
in the generating step there are combined any two original actions that appear sequentially in a control policy that is a result of a reinforcement learning step that was previously applied to the set of original actions of the action library.

6 . The method according to claim 1 , wherein
when control policy learning for the set of actions including the original actions and the combined actions is performed by applying the reinforcement learning algorithm knowledge about a control policy generated on the basis of the original actions only is used.

7 . The method according to claim 1 , wherein
for the application of the reinforced learning algorithm reward functions are used that favor combined actions or faster task achievement.

8 . The method according to claim 1 , wherein
at least the steps of generating the combined actions, storing the combined actions and applying the reinforcement learning algorithm are performed as a simulation.

9 . The method according to claim 1 , wherein
at least the steps of generating the combined actions, storing the combined actions and applying the reinforcement learning algorithm are performed multiple times wherein in each iteration all actions of the resulting action library form the original actions for a next iteration.

Assignees

• Honda Res Inst Europe Gmbh

Inventors

• Mühlig Manuel
• Gienger Michael
• Hayashi Akinobu

Classifications

• G05B2219/40499

  Reinforcement learning algorithm · CPC title

• B25J9/1656 Primary

  characterised by programming, planning systems for manipulators · CPC title

• B25J9/163 Primary

  learning, adaptive, model based, rule based expert control · CPC title