Generation of operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot

Patent metadata

Field	Value
Publication number	US-12544923-B2
Application number	US-202017799984-A
Country	US
Kind code	B2
Filing date	Feb 25, 2020
Priority date	Feb 25, 2020
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

A control device 1 C includes an operation sequence generation means 37 C. The operation sequence generation means 37 C is configured to generate, based on robot operation information Ir indicating operation characteristics of a robot executing a task and peripheral equipment information Ip indicating operation characteristics of peripheral equipment which delivers or receives an object relating to the task to or from the robot, operation sequences Sra and Spa indicating operations to be executed by the robot and the peripheral equipment, respectively.

First claim

What is claimed is:

1 . A control device comprising
at least one memory storing instructions; and at least one processor configured to execute the instructions to:
generate time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:
robot operation information indicating operation characteristics of the robot executing the task; and
peripheral equipment information indicating operation characteristics of the peripheral equipment that delivers or receives the object relating to the task to or from the robot;

generate control signals to control the robot and the peripheral equipment based on:
the operation sequences; and
states of the robot and the peripheral equipment measured during execution of the operation sequences;

provide the control signals to the robot and the peripheral equipment;
predict states of the robot and the peripheral equipment according to the operation sequences;
recognize an actual progress degree of the operation sequences based on a number of completed subtasks;
determine, based on the actual progress degree and an expected progress degree of the operation sequences at a current time step, whether or not the measured states match the predicted states;
regenerate, upon determining that the measured states do not match predicted states, a control signal to control the robot and/or the peripheral equipment to adjust an operation timing of at least one of the robot or the peripheral equipment, and provide the regenerated control signal to the robot and/or the peripheral equipment;
convert the task into a logical formula based on a temporal logic, by:
setting a number of time steps for completing the task;
determining, from a target logical formula, a plurality of candidate combinations of propositions, the propositions, in each candidate combination, each representing a state at each time step, such that each candidate combination satisfies the target logical formula with the set number of times, each candidate combination being a candidate of a logical formula representing a sequence of operations to be instructed to the robot;
generating a time step logical formula in which the candidate combinations are combined; and
generating, based on the time step logical formula, sequences of subtasks to be executed by the robot and the peripheral equipment as the operation sequences, respectively.

2 . The control device according to claim 1 , wherein the at least one processor is further configured to execute the instructions to:
predict a state of the robot after elapse of a predetermined time in a workspace; and generate the operation sequences based on the predicted state, the robot operation information, and the peripheral equipment operation information.

3 . The control device according to claim 2 , wherein the predetermined time is set to a predicted calculation time required for calculation of the operation sequences.

4 . The control device according to claim 1 , wherein the robot operation information includes at least one of:
operation limit information relating to an operation limit of the robot; subtask information indicating subtasks executable by the robot; and abstract model information relating to a model in which dynamics relating to the robot are abstracted, wherein the peripheral equipment operation information includes at least one of: operation limit information relating to an operation limit of the peripheral equipment; subtask information indicating subtasks executable by the peripheral equipment; and abstract model information relating to a model in which dynamics relating to the peripheral equipment are abstracted.

5 . The control device according to claim 4 , wherein the abstract model information included in the robot operation information includes information indicating a condition of switching the dynamics relating to the robot, and
wherein the abstract model information included in the peripheral equipment operation information includes information indicating a condition of switching the dynamics relating to the peripheral equipment.

6 . The control device according to claim 1 , wherein the peripheral equipment is a conveyor configured to convey the object, and
wherein the robot is a robot at least configured to pick up the object.

7 . The control device according to claim 1 , wherein the at least one processor is configured to execute the instructions to:
display an image obtained by capturing an image of a workspace of the robot on a display device and accept an input specifying the object or a type of the object; and generate the operation sequences based on the input.

8 . A control method performed by a computer and comprising:
generating time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:
robot operation information indicating operation characteristics of the robot executing the task; and
peripheral equipment information indicating operation characteristics of the peripheral equipment that delivers or receives the object relating to the task to or from the robot;
generating control signals to control the robot and the peripheral equipment based on:
the operation sequences; and
states of the robot and the peripheral equipment measured during execution of the operation sequences;
providing the control signals to the robot and the peripheral equipment; predicting states of the robot and the peripheral equipment according to the operation sequences; recognizing an actual progress degree of the operation sequences based on a number of completed subtasks; determining, based on the actual progress degree and an expected progress degree of the operation sequences at a current time step, whether or not the measured states match the predicted states; regenerating, upon determining that the measured states do not match predicted states, a control signal to control the robot and/or the peripheral equipment to adjust an operation timing of at least one of the robot or the peripheral equipment, and provide the regenerated control signal to the robot and/or the peripheral equipment; converting the task into a logical formula based on a temporal logic, by:
setting a number of time steps for completing the task;

determining, from a target logical formula, a plurality of candidate combinations of propositions, the propositions, in each candidate combination, each representing a state at each time step, such that each candidate combination satisfies the target logical formula with the set number of times, each candidate combination being a candidate of a logical formula representing a sequence of operations to be instructed to the robot;
generating a time step logical formula in which the candidate combinations are combined; and
generating, based on the time step logical formula, sequences of subtasks to be executed by the robot and the peripheral equipment as the operation sequences, respectively.

9 . A non-transitory computer readable storage medium storing a program executable by a computer to:
generate time step-based operation sequences indicating operations to be executed by a robot executing a task and peripheral equipment that delivers or receives an object relating to the task to or from the robot, based on:
robot operation information indicating operation characteristics of the robot executing the task; and
peripheral equipment information indicating operation characteristics of the periphe

Assignees

• Nec Corp

Inventors

• WAKAYAMA Hisaya
• Ogawa Masatsugu
• Itou Takehiro
• Satoh Mineto
• Oyama Hiroyuki
• Kami Nobuharu

Classifications

• G05B2219/39102

  Manipulator cooperating with conveyor · CPC title

• G05B19/4182

  manipulators and conveyor only · CPC title

• G05B19/41895

  using automatic guided vehicles [AGV] (control of position or course of AGV's G05D1/00) · CPC title

• G05B2219/40113

  Task planning · CPC title

• B25J13/00

  Controls for manipulators (programme controls B25J9/16) · CPC title