Hand-eye calibration method and system

Patent metadata

Field	Value
Publication number	US-11173609-B2
Application number	US-202016743678-A
Country	US
Kind code	B2
Filing date	Jan 15, 2020
Priority date	Jan 22, 2019
Publication date	Nov 16, 2021
Grant date	Nov 16, 2021

Abstract

A hand-eye calibration system and method are provided. The system includes a robot on which a small pattern is mounted, a camera configured to photograph the robot, a memory, and a processor configured to move the robot, acquire posture information of the moved robot, acquire an image from the camera, move the camera after performing the robot movement, the posture information acquisition, and the image acquisition a first predetermined number of times, and perform hand-eye calibration for the robot based on the posture information and the images, which are obtained by repeatedly performing of the robot movement, the posture information acquisition, the image acquisition, and the camera movement.

First claim

What is claimed is:

1. A hand-eye calibration system, comprising:
a robot on which a small pattern is mounted;
a camera configured to photograph the robot;
a memory; and
a processor configured to:
move the robot,
acquire posture information of the moved robot,
acquire an image from the camera,
move the camera after performing the robot movement, the posture information acquisition, and the image acquisition a first predetermined number of times, and
perform hand-eye calibration for the robot based on the posture information and the images, which are obtained by repeatedly performing of the robot movement, the posture information acquisition, the image acquisition, and the camera movement.

2. The hand-eye calibration system of claim 1 , wherein the robot includes two-arms, and
wherein the small pattern is mounted on a first arm of the robot, and the camera is mounted on a second arm of the robot.

3. The hand-eye calibration system of claim 1 , wherein the memory stores a traveling direction within a checkerboard, and
wherein the processor is further configured to move the robot on a plane based on the traveling direction.

4. The hand-eye calibration system of claim 3 , wherein the processor is further configured to perform the hand-eye calibration by matching first coordinate information of a small pattern included in the acquired image and second coordinate information of the small pattern in the checkerboard.

5. The hand-eye calibration system of claim 3 , wherein the processor is further configured to move the robot in X-axis units or Y-axis units based on the traveling direction.

6. The hand-eye calibration system of claim 3 , wherein the small pattern mounted on the robot is configured in a form of a smaller checkerboard than the checkerboard.

7. The hand-eye calibration system of claim 1 , wherein the processor is further configured to initialize positions of the robot and the camera after performing the robot movement, the posture acquisition, the image acquisition the first predetermined number of times.

8. The hand-eye calibration system of claim 7 , wherein the processor is further configured to:
after the initialization, determine whether the position of the camera is to be moved;
move the camera, in response to determining that the position of the camera is to be moved; and
perform the robot movement, the posture information acquisition, and the image acquisition the first predetermined number of times.

9. The hand-eye calibration system of claim 8 , wherein the processor is further configured to:
determine whether the position of the camera has been moved less than a second predetermined number of times; and
when the position of the camera has been moved less than the second predetermined number of times, determine that the position of the camera is to be moved.

10. The hand-eye calibration system of claim 1 , wherein the processor is further configured to:
monitor the robot after performing the hand-eye calibration; and
diagnose a fault of the robot based on the monitoring.

11. The hand-eye calibration system of claim 10 , wherein the processor is further configured to:
acquire first pattern coordinate system information based on a pattern position within the image acquired from the camera;
acquire second pattern coordinate system information obtained by robot kinematics; and
diagnose a fault of the robot based on the first pattern coordinate system information and the second pattern coordinate system information.

12. The hand-eye calibration system of claim 11 , wherein the processor is further configured to:
identify whether a difference between the first pattern coordinate system information and the second pattern coordinate system information is smaller than a threshold value;
determine that the robot operates normally when the difference is smaller than the threshold value; and
determine that the robot does not operate normally when the difference is greater than or equal to the threshold value.

13. The hand-eye calibration system of claim 10 , wherein when the robot is diagnosed as having the fault, the processor is further configured to re-execute the hand-eye calibration.

14. A method for operating a hand-eye calibration system including a robot on which a small pattern is mounted and a camera configured to photograph the robot, the method comprising:
moving the robot;
acquiring posture information of the moved robot;
acquiring an image from the camera;
moving the camera after performing the robot movement, the posture information acquisition, and the image acquisition a first predetermined number of times; and
performing hand-eye calibration for the robot based on the posture information and the image, which are obtained by repeating the robot movement, the posture information acquisition, the image acquisition, and the camera movement.

15. The method of claim 14 , wherein the robot includes two-arms, and
wherein the small pattern is mounted on a first arm of the robot, and the camera is mounted on a second arm of the robot.

16. The method of claim 14 , wherein moving the robot comprises:
storing a traveling direction within a checkerboard in a memory of the hand-eye calibration system; and
moving the robot in X-axis units or Y-axis units based on the traveling direction.

17. The method of claim 16 , wherein performing the hand-eye calibration comprises:
matching first coordinate information of a small pattern included in the acquired image and second coordinate information of the small pattern in the checkerboard.

18. The method of claim 14 , further comprising:
initializing positions of the robot and the camera after performing the robot movement, the posture information acquisition, and the image acquisition the first predetermined number of times;
determining whether the position of the camera is to be moved after the initialization;
moving the camera in response to determining that the position of the camera is to be moved; and
performing the robot movement, the posture information acquisition, and the image acquisition the first predetermined number of times.

19. The method of claim 14 , further comprising:
monitoring the robot after performing the hand-eye calibration; and
diagnosing a fault of the robot based on the monitoring.

20. The method of claim 19 , wherein diagnosing the fault of the robot comprises:
acquiring first pattern coordinate system information based on a pattern position in the image acquired from the camera;
acquiring second pattern coordinate system information obtained by robot kinematics; and
diagnosing the fault of the robot based on the first pattern coordinate system information and the second pattern coordinate system information.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Sohee
• Yoon Sukjune
• Kang Yuso
• Roh Kyungshik
• Bahn Wook
• Choi Minyong

Classifications

• G05B2219/39008

  Fixed camera detects reference pattern held by end effector · CPC title

• B25J9/1692 Primary

  Calibration of manipulator · CPC title

• G05B2219/39057

  Hand eye calibration, eye, camera on hand, end effector · CPC title

• B25J19/04

  Viewing devices · CPC title

• B25J19/023

  including video camera means · CPC title