Image processing device, method of controlling the same, and recording medium

Patent metadata

Field	Value
Publication number	US-2020167886-A1
Application number	US-201916684954-A
Country	US
Kind code	A1
Filing date	Nov 15, 2019
Priority date	Nov 26, 2018
Publication date	May 28, 2020
Grant date	—

Abstract

A work system includes a robot control device, a robot, an image processing device, and an imaging device. The robot control device controls an operation of the robot using a teaching point. The image processing device acquires an image by controlling the imaging device and capturing an image of a workpiece at the teaching point, and stores image data after imaging in association with data of an operation position of the robot corresponding to the teaching point. When a user performs work restoration, the image processing device performs a process of assisting with setting of work restoration by performing control for acquiring an image captured by the imaging device at the operation position of the robot during work restoration and displaying a difference between the image and the stored image.

First claim

What is claimed is:

1 . An image processing device comprising:
at least one processor and memory holding a program which makes the processor function as: a control unit configured to control an operation of a device for work using a teaching point representing a position or posture of the device for work; a storage unit configured to store image data at the teaching point captured by an imaging unit and data of the position or posture of the device for work corresponding to the teaching point in association with each other; and an image processing unit configured to perform an assistance process of assisting with setting of work restoration by processing data of a first image captured by the imaging unit at the teaching point of the device for work when work restoration is performed and data of a second image at the teaching point which is read out from the storage unit, and displaying the first and second images.

2 . The image processing device according to claim 1 , wherein the image processing unit performs the assistance process by extracting features of the first and second images, calculating an amount of shift between positions or postures of the feature of the first image and the feature of the second image, and displaying a rate of restoration of work restoration.

3 . The image processing device according to claim 2 , wherein the image processing unit performs the assistance process by displaying a rate of concordance or a degree of similarity between the feature of the first image and the feature of the second image as the rate of restoration.

4 . The image processing device according to claim 2 , wherein the image processing unit performs the assistance process by obtaining and displaying a correction direction for improving the rate of restoration using the amount of shift.

5 . The image processing device according to claim 1 , wherein the image processing unit performs control for displaying a setting screen having an image display region in the assistance process and displaying the first and second images in a superimposed manner in the image display region.

6 . The image processing device according to claim 5 , wherein the image processing unit performs control for displaying feature information of an image extracted through image processing in the image display region together with the first and second images.

7 . The image processing device according to claim 5 , wherein the image processing unit performs control for comparing feature information of a captured image during work of the device for work in a time-series manner to estimate the feature information, and displaying an extraction range of the feature information in the image display region.

8 . The image processing device according to claim 5 , wherein the image processing unit performs a process of displaying a region having a feature amount of the first image larger than a threshold in the image display region, and accepting adjustment of work restoration or a change of setting.

9 . The image processing device according to claim 2 , wherein, while an image processing program is being executed during work of the device for work, the image processing unit performs a process of stopping execution of the image processing program in a case where the amount of shift falls outside a designated error amount.

10 . The image processing device according to claim 9 , wherein the image processing unit outputs a processing result of graphical representation or trend analysis by repeatedly executing the image processing program and monitoring a feature extraction result of an image or a change of the amount of shift.

11 . A method of controlling an image processing device, comprising:
controlling an operation of a device for work using a teaching point representing a position or posture of the device for work; performing control for capturing an image of a subject at the teaching point using an imaging unit; storing image data at the teaching point captured by the imaging unit and data of the position or posture of the device for work corresponding to the teaching point in association with each other in a storage unit; and performing a process of assisting with setting of work restoration by processing data of a first image captured by the imaging unit at the teaching point of the device for work when work restoration is performed and data of a second image at the teaching point which is read out from the storage unit, and displaying the first and second images.

12 . A non-transitory recording medium storing a control program of an image processing device causing a computer to perform each step of a control method of the image processing device, the method comprising:
controlling an operation of a device for work using a teaching point representing a position or posture of the device for work; performing control for capturing an image of a subject at the teaching point using an imaging unit; storing image data at the teaching point captured by the imaging unit and data of the position or posture of the device for work corresponding to the teaching point in association with each other in a storage unit; and performing a process of assisting with setting of work restoration by processing data of a first image captured by the imaging unit at the teaching point of the device for work when work restoration is performed and data of a second image at the teaching point which is read out from the storage unit, and displaying the first and second images.

Assignees

• Canon Kk

Inventors

• Cho Genki
• MIYAKE Yuuya

Classifications

• G06T1/0014 Primary

  Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023) · CPC title

• B25J9/1697

  Vision controlled systems · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title

• H04N23/633 Primary

  for displaying additional information relating to control or operation of the camera · CPC title

• G06T7/33 Primary

  using feature-based methods · CPC title