Control device, control system, robot system, and control method

Patent metadata

Field	Value
Publication number	US-12343873-B2
Application number	US-202017776602-A
Country	US
Kind code	B2
Filing date	Nov 16, 2020
Priority date	Nov 15, 2019
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

A control device includes: first circuitry that generates a command to cause a robot to autonomously grind a grinding target portion; second circuitry that generates a command to cause the robot to grind a grinding target portion according to manipulation information from an operation device; third circuitry that controls operation of the robot according to the command; storage that stores image data of a grinding target portion and operation data of the robot corresponding to the command; and forth circuitry that performs machine learning by using image data of a grinding target portion and the operation data for the grinding target portion, receives the image data as input data, and outputs an operation correspondence command corresponding to the operation data as output data. The first circuitry generates the command, based on the operation correspondence command.

First claim

The invention claimed is:

1. A control device for a robot that performs grinding work by using a grinder, the control device comprising:
autonomous command generation circuitry that generates an autonomous operation command for causing the robot to autonomously grind a grinding target portion;
manual command generation circuitry that generates a manual operation command for causing the robot to grind a grinding target portion according to manipulation information outputted from an user interface for operating the robot;
operation control circuitry that controls operation of the grinding work of the robot according to the operation command;
storage that stores image data of a grinding target portion and operation data of the robot corresponding to the operation command generated for the grinding target portion in a state of the image data; and
learning circuitry that performs machine learning by using image information that is information regarding image data of a grinding target portion and the operation data of the robot for the grinding target portion in a state of the image information, receives the image information of a grinding target portion as input data, and outputs an operation correspondence command corresponding to the operation data of the robot for the grinding target portion in a state of the image information as output data, wherein;
the operation correspondence command is a command including:
a force command that commands magnitude and a direction of force that is applied to a grinding target portion from the grinder by the robot, and
a position command that commands a three-dimensional position and three-dimensional posture of the grinder which are realized by the robot, and

the autonomous command generation circuitry generates the autonomous operation command, based on the operation correspondence command of the learning circuitry.

2. The control device according to claim 1 , wherein:
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the manual operation command generated for the grinding target portion in a state of the image information.

3. The control device according to claim 1 , wherein;
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the autonomous operation command and the manual operation command that are generated for the grinding target portion in a state of the image information.

4. The control device according to claim 1 , wherein:
during operation of the robot according to the autonomous operation command, the manual command generation circuitry receives input of correction manipulation information that is the manipulation information for correcting the operation of the robot, via the user interface, and generates a correction manual operation command that is the manual operation command, according to the correction manipulation information, and
the learning circuitry performs machine learning by using the image information of a grinding target portion and the operation data of the robot corresponding to the autonomous operation command and the manual operation command that are generated for the grinding target portion in a state of the image information.

5. The control device according to claim 1 , wherein:
the autonomous command generation circuitry generates:
a first autonomous operation command that is the autonomous operation command for first grinding work in which the robot autonomously grinds a grinding target portion according to a predetermined operation procedure, and
a second autonomous operation command that is the autonomous operation command for second grinding work in which the robot autonomously grinds the grinding target portion, based on the operation correspondence command of the learning circuitry after the first grinding work, and

the autonomous command generation circuitry generates the second autonomous operation command, based on the operation correspondence command outputted by the learning circuitry with the image information of the grinding target portion as input data.

6. The control device according to claim 1 , wherein;
the autonomous command generation circuitry generates a third autonomous operation command that is the autonomous operation command for third grinding work in which an unground grinding target portion is autonomously ground, and
the autonomous command generation circuitry generates the third autonomous operation command, based on the operation correspondence command outputted by the learning circuitry with the image information of the grinding target portion as input data.

7. The control device according to claim 1 , wherein the learning circuitry performs machine learning and input/output by using a learning model including a neural network.

8. The control device according to claim 1 , wherein the learning circuitry uses data representing three-dimensional positions of various portions on a grinding target portion, as the image information.

9. The control device according to claim 8 , further comprising:
first image processing circuitry that detects three-dimensional positions of various portions on a grinding target portion by processing an image of the grinding target portion, and generates image data representing the three-dimensional positions of said various portions.

10. The control device according to claim 1 , further comprising:
first image processing circuitry that, based on at least either one of a shade or a hue represented in an image of a grinding target portion taken by using an imager including a non-reflective light source and a camera, detects a distance from the camera to various positions on the grinding target portion, and generates image data representing the distance to said various positions.

11. The control device according to claim 1 , further comprising:
imaging control circuitry that controls operation of an imager that takes an image of a grinding target portion, and acquires image data of the grinding target portion,
wherein:
the imaging control circuitry causes the imager to take an image of a grinding target portion at a timing before grinding work on the grinding target portion and a predetermined timing during the grinding work, and
the learning circuitry outputs the operation correspondence command with the image information regarding the image data acquired by the imaging control circuitry as input data.

12. The control device according to claim 11 , wherein:
the operation control circuitry causes the robot to grind all grinding target portions of a grinding target group including the grinding target portions according to the autonomous operation command,
the imaging control circuitry causes the imager to take images of said all grinding target portions after grinding work according to the autonomous operation command on said all grinding target portions is completed, and
the control device causes a display to present the images of said all grinding target portions.

13. The control device according to claim 11 , further comprising:
second image processing circuitry,
wherein the second image processing circuitry processes image data of a grinding target portion taken by the imager, generates image data representing a three-dimensional shape of the grinding target portion by using at least either one of a color, a pattern, or a shade, and causes the display to present the image data.

14. The control device according to claim 1 , further comprising:
movement con

Assignees

• Kawasaki Heavy Ind Ltd

Inventors

• Yonemoto Shingo
• Kozuki Takanori
• Akamatsu Masahiko
• Hasunuma Hitoshi
• Kamon Masayuki

Classifications

• B25J13/085

  Force or torque sensors (B25J13/082, B25J13/084 take precedence) · CPC title

• B25J11/0065

  Polishing or grinding · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• B25J9/1679

  characterised by the tasks executed · CPC title

• B25J9/1664 Primary

  characterised by motion, path, trajectory planning · CPC title