Monitoring devices, monitored control systems and methods for programming such devices and systems

Patent metadata

Field	Value
Publication number	US-11042149-B2
Application number	US-201716479944-A
Country	US
Kind code	B2
Filing date	Mar 1, 2017
Priority date	Mar 1, 2017
Publication date	Jun 22, 2021
Grant date	Jun 22, 2021

Abstract

Monitored control system includes an action device and a monitoring device for monitoring the operation of the action device. The action device includes actuator modules, a sensor module and a control module to command the actuator modules to perform a predefined task using the sensor data. The monitoring device receive monitored data from the action device and compute an action device operation indicator indicative of a correct or abnormal operation of the action device on the basis of the monitored data. The action device operation indicator is computed on the basis of a learned value of an adjustable monitoring parameter. The learned value is determined from a set of labelled data generated from trial datasets recorded by the action device during an iterative learning procedure.

First claim

The invention claimed is:

1. A monitoring device for monitoring an operation of an action device, the action device comprising:
a plurality of actuator modules respectively configured to be commanded by a plurality of respective command signals;
at least one sensor module configured to acquire sensor data;
a communication module; and
a control module configured to receive the sensor data and to generate a plurality of command signals to command the actuator modules to perform a predefined task,
the monitoring device comprising:
a communication module configured to communicate with the communication module of the action device to receive monitored data from the action device, the monitored data comprising at least one of the command signals and the sensor data; and
a control module configured to compute an action device operation indicator indicative of a correct or abnormal operation of the action device based on the monitored data and at least one learned value of at least one adjustable monitoring parameter, wherein:
the learned value of the at least one adjustable monitoring parameter is determined based on a set of labelled data generated at least in part from a plurality of trial datasets recorded by the action device during an iterative learning procedure;
the monitoring device further comprises a monitoring device learning module configured to:
receive the plurality of trial datasets and at least one test dataset from the action device;
label the trial datasets with at least a first label and the at least one test dataset with at least a second label; and
determine the learned value of the at least one adjustable monitoring parameter by training a classifier to respectively classify the plurality of trial datasets with the at least one first label and the at least one test dataset with the at least one second label;

the plurality of command signals is generated based on the sensor data and a learned value of at least one adjustable control parameter, the learned value of the at least one adjustable control parameter being determined by an iterative learning procedure wherein the action device performs a plurality of task trials with at least two different values of the at least one adjustable control parameter while recording an associated plurality of the trial datasets, each trial dataset comprising at least command signals generated to command the actuator modules during a task trial and sensor data acquired by the sensor module while the actuator modules are commanded; and
the set of labelled data is generated from:
the trial datasets of the action device recorded during the iterative learning procedure of the action device; and
the at least one test dataset, wherein the at least one test dataset is recorded by setting the at least one adjustable control parameter of the action device to the learned value of the at least one adjustable control parameter and commanding the action device to perform, at least once, the predefined task while recording the at least one test dataset.

2. The monitoring device according to claim 1 , wherein the control module of the monitoring device computes the action device operation indicator using a classifier, wherein the learned value of the at least one adjustable monitoring parameter is a parameter of the classifier.

3. A monitored control system comprising:
the monitoring device according to claim 1 ; and
the action device, comprising:
the plurality of actuator modules respectively configured to be commanded by the plurality of respective command signals;
the at least one sensor module configured to acquire capable of acquiring the sensor data;
the communication module, and
the control module configured to receive the sensor data and to generate the plurality of command signals to command the actuator modules to perform the predefined task.

4. The monitored control system according to claim 3 , wherein the sensor module of the action device comprises:
at least one input sensor configured to acquire input sensor data used by the control module of the action device to generate the plurality of command signals; and
at least one status sensor configured to acquire status sensor data used by the control module of the monitoring device to compute the action device operation indicator.

5. The monitored control system according to claim 4 , wherein
the input sensor comprises a bi-dimensional or tri-dimensional sensor such as a camera, and
the status sensor comprises a lower dimensional sensor such as a vibration sensor, noise sensor, temperature sensor, position sensor, orientation sensor or power sensor.

6. The monitored control system according to claim 3 , further comprising an action device learning module configured to:
command the control module of the action device to perform the plurality of predefined task trials with at least two different values of at least one adjustable control parameter and at least one predefined task with a learned value of the at least one adjustable control parameter,
respectively record the plurality of trial datasets and the at least one test dataset, each trial dataset and test dataset comprising command signals generated to command the actuator modules during a respective predefined task trial and predefined task and sensor data acquired by the sensor module while the actuator modules are commanded, and
determine a learned value of at least one adjustable control parameter of the control module of the action device by an iterative learning procedure based on the plurality of trial datasets.

7. A method for programming the monitoring device according to claim 1 , the method comprising:
a monitoring device learning operation wherein a learned value of at least one adjustable monitoring parameter of the monitoring device is determined by
labelling the trial datasets with at least a first label and at least one test dataset with at least a second label; and
determining the learned value of at least one adjustable monitoring parameter by training a classifier to classify the trial datasets with the at least one first label and the test datasets with the at least one second label.

8. A method for programming the monitored control system according to claim 3 , the method comprising:
an action device learning operation wherein a learned value of at least one adjustable control parameter of the control module of the action device is determined by an iterative learning procedure comprising commanding the action device to perform a plurality of predefined task trials with at least two different values of the at least one adjustable control parameter while recording the plurality of trial datasets, each trial dataset comprising command signals generated to command the actuator modules during a predefined task trial and sensor data acquired by the sensor module while the actuator modules are commanded;
an action device test operation wherein at least one test dataset is recorded by:
setting the at least one adjustable control parameter to the learned value determined during the action device learning operation; and
commanding the action device to perform, at least once, the predefined task while recording the least one test dataset, the at least one test dataset comprising command signals generated to command the actuator modules during said predefined task and sensor data acquired by the sensor module while the actuator modules are commanded; and

performing a method for programming the monitoring device.

9. The method according to claim 8 , wherein
during the action device learning operation, the learning level indicator is associated with each trial dataset and is indicative of the deviation of the task trial from the

Assignees

• Omron Tateisi Electronics Co

Inventors

• Ando Tanichi

Classifications

• G06V2201/06

  Recognition of objects for industrial automation · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• Y02P90/02

  Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] · CPC title

• G05B13/0265

  the criterion being a learning criterion · CPC title

• G05B2219/31483

  Verify monitored data if valid or not by comparing with reference value · CPC title