Automatic Operation Control Method and System

Patent metadata

Field	Value
Publication number	US-2020333775-A1
Application number	US-202016845522-A
Country	US
Kind code	A1
Filing date	Apr 10, 2020
Priority date	Apr 17, 2019
Publication date	Oct 22, 2020
Grant date	—

Abstract

An object of the present invention is to reduce an error between an actual machine and a simulation by removing the influence of overlearning of an adjustment by a mathematically-described function, and to optimize automatic operation control of the machine. An automatic operation control system for controlling an automatic operation of a machine sets a first model showing a relation between a control signal string input to the machine on the basis of a mathematically-described function and data output from the machine controlled in accordance with the control signal string. In a learning process including learning the automatic operation control of the machine, the system executes learning using the first model until a first condition is satisfied. After the first condition is satisfied, the learning is executed using a second model that is a model after the first model is changed one or more times until a second condition meaning overlearning is satisfied or the learning is finished without satisfying the second condition.

First claim

What is claimed is:

1 . An automatic operation control system for controlling an automatic operation of a machine, comprising:
a question setting unit that sets a first model showing a relation between a control signal string input to the machine on the basis of a mathematically-described function and data output from the machine controlled in accordance with the control signal string; a learning execution unit that executes a learning process including to learn automatic operation control of the machine; and an operation control unit that controls the automatic operation of the machine by inputting a control signal string in accordance with the result of the learning process into the machine, wherein in the learning process, the learning execution unit executes learning using the first model until a first condition is satisfied, and executes, after the first condition is satisfied, learning using a second model that is a model after the first model is changed one or more times until a second condition meaning overlearning is satisfied or the learning is finished without satisfying the second condition.

2 . The automatic operation control system according to claim 1 ,
wherein the second model is a model as a result of applying to the first model a predetermined ratio of a mathematically-described third model that is different from the first model.

3 . The automatic operation control system according to claim 2 , comprising a reliability setting unit that calculates first reliability on the basis of a first error between first simulation result data output from the first model into which a first control signal string has been input and first real world data output from the machine into which the first control signal string has been input,
wherein the predetermined ratio is a ratio smaller than the calculated first reliability.

4 . The automatic operation control system according to claim 3 ,
wherein the first condition is a condition based on the first reliability and the number times of learning.

5 . The automatic operation control system according to claim 3 , comprising an applied model generation unit that generates the third model that is a model to which the first model is adjusted so that the first error falls within an allowable error range.

6 . The automatic operation control system according to claim 1 ,
wherein in the case where the second condition is satisfied, the learning execution unit finishes the learning process.

7 . The automatic operation control system according to claim 3 ,
wherein the reliability setting unit displays at least one of the first error and the first reliability, and wherein in the case where permission of the learning process has been accepted for the display, the learning execution unit executes the learning process.

8 . The automatic operation control system according to claim 1 ,
wherein both of the learning using the first model and the learning using the second model are reinforcement learning, wherein the second condition is at least one of the followings: a value according to a reward obtained in the reinforcement learning using the second model is larger than that according to a reward obtained in the reinforcement learning using the first model; and the fluctuation range of the reward obtained in the reinforcement learning using the second model exceeds that of the reward obtained in the reinforcement learning using the first model.

9 . The automatic operation control system according to claim 3 , comprising a reliability/first model update unit that executes, in the case where second reliability on the basis of a second error between second simulation result data output from the second model by inputting a second control signal string to the second model in the case where the learning process has been finished without satisfying the second condition and second real world data output from the machine into which the second control signal string has been input exceeds the first reliability, a reliability/first model update process that includes updating the second reliability to new first reliability and updating to a new first model a model as a result of applying the third model with a ratio based on the new first reliability to the first model,
wherein the new first model is used in the learning process for each reliability/first model update process.

10 . The automatic operation control system according to claim 3 , comprising an evaluation unit that displays at least one of second simulation result data output from the second model by inputting the second control signal string to the second model in the case where the learning process has been finished without satisfying the second condition and the first reliability, and inputs the second control signal string to the machine in the case where permission of actual operation confirmation of the machine has been accepted for the display.

11 . The automatic operation control system according to claim 10 , comprising a reliability/first model update unit that executes, in the case where second reliability on the basis of a second error between second real world data output from the machine into which the second control signal string has been input and second simulation result data exceeds the first reliability, a reliability/first model update process that includes updating the second reliability to new first reliability and updating to anew first model a model as a result of applying the third model with a ratio based on the new first reliability to the first model,
wherein the new first model is used in the learning process for each reliability/first model update process.

12 . The automatic operation control system according to claim 1 ,
wherein the machine is an industrial machine.

13 . An automatic operation control method for controlling an automatic operation of a machine, comprising the steps of:
setting a first model showing a relation between a control signal string input to the machine on the basis of a mathematically-described function and data output from the machine controlled in accordance with the control signal string; executing a learning process including to learn automatic operation control of the machine; and controlling the automatic operation of the machine by inputting a control signal string in accordance with the result of the learning process into the machine, wherein in the learning process, learning is executed using the first model until a first condition is satisfied, and after the first condition is satisfied, learning is executed using a second model that is a model after the first model is changed one or more times until a second condition meaning overlearning is satisfied or the learning is finished without satisfying the second condition.

Assignees

• Hitachi Ltd

Inventors

• Matsumoto Morio
• Kudo Fumiya
• Kujirai Toshihiro

Classifications

• G05B13/042 Primary

  in which a parameter or coefficient is automatically adjusted to optimise the performance · CPC title

• G05B2219/40499

  Reinforcement learning algorithm · CPC title

• G05B17/02 Primary

  electric · CPC title

• G05B2219/40298

  Manipulator on vehicle, wheels, mobile · CPC title

• B25J9/1671

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title