Machine learning apparatus, servo control apparatus, servo control system, and machine learning method

Patent metadata

Field	Value
Publication number	US-10747193-B2
Application number	US-201815997043-A
Country	US
Kind code	B2
Filing date	Jun 4, 2018
Priority date	Jun 22, 2017
Publication date	Aug 18, 2020
Grant date	Aug 18, 2020

Abstract

To perform reinforcement learning enabling to prevent complicated adjustment of coefficients of backlash compensation and backlash acceleration compensation. A machine learning apparatus includes a state information acquiring part for acquiring, from a servo control apparatus, state information including at least position deviation and a set of coefficients to be used by a backlash acceleration compensating part, by making the servo control apparatus execute a predetermined machining program, an action information output part for outputting action information including adjustment information on the set of coefficients included in the state information to the servo control apparatus, a reward output part for outputting a reward value in the reinforcement learning on the basis of the position deviation included in the state information, and a value function updating part for updating an action-value function on the basis of the reward value output by the reward output part, the state information and the action information.

First claim

What is claimed is:

1. A machine learning apparatus for performing reinforcement learning to a servo control apparatus using a backlash compensation parameter and a backlash acceleration compensation parameter, the servo control apparatus creating a backlash compensation value compensating a position command or a position deviation, the servo control apparatus creating a backlash acceleration compensation value compensating a speed command, the machine learning apparatus comprising:
a processor; and
a non-transitory memory having stored thereon executable instructions, which when executed, cause the processor to perform:
outputting, to the servo control apparatus, action information including adjustment information on the backlash compensation parameter and the backlash acceleration compensation parameter;
acquiring, from the servo control apparatus, state information including position deviation and the backlash compensation parameter and the backlash acceleration compensation parameter, the position deviation being obtained from the position command and a fed-back position, at a time of making the servo control apparatus execute a predetermined machining program on the basis of the action information;
outputting a reward value in the reinforcement learning on the basis of the position deviation included in the state information; and
updating an action-value function on the basis of the reward value, the state information, and the action information,

wherein the reinforcement learning is performed for the backlash compensation parameter, and then the reinforcement learning is performed for the backlash acceleration compensation parameter.

2. The machine learning apparatus according to claim 1 , wherein the processor outputs the reward value on the basis of an absolute value of the position deviation.

3. The machine learning apparatus according to claim 1 ,
wherein the executable instructions further cause the processor to perform generating and outputting the backlash compensation parameter and the backlash acceleration compensation parameter on the basis of the updated action-value function.

4. The servo control apparatus including the machine learning apparatus according to claim 1 .

5. A servo control system including the machine learning apparatus and the servo control apparatus according to claim 1 .

6. A machine learning method for a machine learning apparatus to perform reinforcement learning to a servo control apparatus using a backlash compensation parameter and a backlash acceleration compensation parameter, the servo control apparatus creating a backlash compensation value compensating a position command or a position deviation, the servo control apparatus creating a backlash acceleration compensation value compensating a speed command, the machine learning method comprising:
outputting, to the servo control apparatus, action information including adjustment information on the backlash compensation parameter and the backlash acceleration compensation parameter;
acquiring, from the servo control apparatus, state information including position deviation and the backlash compensation parameter and the backlash acceleration compensation parameter, the position deviation being obtained from the position command and a fed-back position, at a time of making the servo control apparatus execute a predetermined machining program on the basis of the action information;
outputting a reward value in the reinforcement learning on the basis of the position deviation included in the state information; and
updating an action-value function on the basis of the reward value, the state information, and the action information,

wherein the reinforcement learning is performed for the backlash compensation parameter, and then the reinforcement learning is performed for the backlash acceleration compensation parameter.

Assignees

• Fanuc Corp

Inventors

• Ikai Satoshi
• Nakamura Tsutomu
• Aizawa Tomoyuki

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N20/00

  Machine learning · CPC title