Control device, control method and storage medium

Patent metadata

Field	Value
Publication number	US-12318939-B2
Application number	US-202017799717-A
Country	US
Kind code	B2
Filing date	Feb 25, 2020
Priority date	Feb 25, 2020
Publication date	Jun 3, 2025
Grant date	Jun 3, 2025

Abstract

A control device 1 A mainly includes a display control means 15 A and an operation sequence generation means 16 A. The display control means 15 A is configured to transmit display information S 2 relating to a task to be executed by a robot to a display device 2 A. The operation sequence generation means 16 A is configured, in a case that the display control means 15 A has received, from the display device 2 A, task designation information that is input information which schematically specifies the task, to generate an operation sequence to be executed by the robot based on the task designation information Ia.

First claim

What is claimed is:

1. A control device comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
transmit, to a display device, display information including a workspace image of a workspace of a robot which executes a task;
upon receiving, from the display device, task designation information that is input information which specifies a position or a destination of an object for the execution of the task:
convert an objective task specified by the task designation information, into a logical formula based on a temporal logic;
generate, from the logical formula, a time step logical formula representing states at each time step for completing the objective task; and
generate an operation sequence to be executed by the robot, based on the task designation information and based on the time step logical formula; and

control the robot to execute the operation sequence that has been generated.

2. The control device according to claim 1 ,
wherein the at least one processor is configured to execute the instructions to determine the task based on the task designation information generated with reference to the workspace image.

3. The control device according to claim 1 ,
wherein the at least one processor is configured to execute the instructions to,
in a case of receiving, from the display device, recognition assistance information for assisting recognition of the object,
generate the operation sequence by recognizing the object based on the recognition assistance information.

4. The control device according to claim 3 ,
wherein the at least one processor is configured to execute the instructions to recognize the object based on the recognition assistance information generated with reference to the workspace image.

5. The control device according to claim 4 ,
wherein the at least one processor is configured to execute the instructions to acquire the recognition assistance information and the workspace image as training data to be used for learning an inference engine configured to perform inference regarding the object.

6. The control device according to claim 1 ,
wherein the at least one processor is configured to execute the instructions to communicate the display information and the input information with the display device via a communication network in which communication quality is not guaranteed.

7. The control device according to claim 1 ,
wherein,
after transmitting display information on a selection view for selecting the robot subject to generation of the operation sequence,
the at least one processor is configured to execute the instructions to transmit, to the display device, display information on a task input view for specifying the task selected on the selection view.

8. The control device according to claim 1 ,
wherein the at least one processor is configured to execute the instructions to
set a proposition to be used in the logical formula, the proposition corresponding to an abstract state of the object.

9. A control method performed by a computer and comprising:
transmitting, to a display device, display information including a workspace image of a workspace of a robot which executes a task;
upon receiving, from the display device, task designation information that is input information which specifies a position or a destination of an object for the execution of the task;
converting an objective task specified by the task designation information, into a logical formula based on a temporal logic;
generating, from the logical formula, a time step logical formula representing states at each time step for completing the objective task; and
generating an operation sequence to be executed by the robot, based on the task designation information and based on the time step logical formula; and
controlling the robot to execute the operation sequence that has been generated.

10. A non-transitory computer readable storage medium storing a program executable by a computer to perform processing comprising:
transmitting, to a display device, display information including a workspace image of a workspace of a robot which executes a task;
upon receiving, from the display device, task designation information that is input information which specifies a position or a destination of an object for the execution of the task;
converting an objective task specified by the task designation information, into a logical formula based on a temporal logic;
generating, from the logical formula, a time step logical formula representing states at each time step for completing the objective task; and
generating an operation sequence to be executed by the robot, based on the task designation information and based on the time step logical formula; and
controlling the robot to execute the operation sequence that has been generated.

Assignees

• Nec Corp

Inventors

• Ogawa Masatsugu
• WAKAYAMA Hisaya
• Itou Takehiro
• Satoh Mineto
• Oyama Hiroyuki
• Kami Nobuharu

Classifications

• B25J9/163

  learning, adaptive, model based, rule based expert control · CPC title

• B25J9/1671

  characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems · CPC title

• G05B2219/40113

  Task planning · CPC title

• G05B2219/40115

  Translate goal to task program, use of expert system · CPC title

• G05B2219/40336

  Optimize multiple constraints or subtasks · CPC title