Robot and electronic device for performing hand-eye calibration

Patent metadata

Field	Value
Publication number	US-10780585-B2
Application number	US-201815860071-A
Country	US
Kind code	B2
Filing date	Jan 2, 2018
Priority date	Jan 4, 2017
Publication date	Sep 22, 2020
Grant date	Sep 22, 2020

Abstract

A robot for performing hand-eye calibration is provided. The robot includes a robot arm including a plurality of joints, a plurality of arm sections, and an end effector, a communication interface, and a control circuit. The control circuit controls the robot arm to place the external object on a worktable after the external object is grasped by the end effector, acquires coordinates of a central point of the external object in a coordinate system of the camera from an image of the external object, and calculates a calibration parameter for defining a relation between a coordinate system of the end effector and the coordinate system of the camera, based on coordinates of the end effector in a base coordinate system of the robot and coordinates of the central point of the external object in the coordinate system of the camera when the external object is placed on the worktable.

First claim

What is claimed is:

1. A robot for performing hand-eye calibration, the robot comprising:
a robot arm including:
a plurality of joints,
a plurality of arm sections rotatably coupled to each other by at least a part of the plurality of joints,
an end effector connected with one of the plurality of joints and configured to grasp an external object, the end effector having a coordinate system, and
a base coordinate system;

a communication interface to connect with a camera in a wireless or wired manner, the camera having a coordinate system and positioned outside the robot arm; and
a control circuit electrically connected with the robot arm and the communication interface,
wherein the control circuit is configured to:
control the robot arm to place the external object on a worktable after the external object is grasped by the end effector such that a center of the end effector coincides with a center of the external object,
control the robot arm to move out of sight of the camera without grasping the external object placed on the worktable, before an image of the external object is captured by the camera,
acquire coordinates of a central point of the external object in the coordinate system of the camera from an image of the external object captured by the camera, and
calculate a calibration parameter for defining a relation between the coordinate system of the end effector and the coordinate system of the camera, based on coordinates of the end effector in the base coordinate system of the robot and coordinates of the central point of the external object in the coordinate system of the camera, when the external object is placed on the worktable.

2. The robot of claim 1 ,
wherein the robot further includes the camera, and
wherein the communication interface is an interface which connects the camera with the control circuit.

3. The robot of claim 1 , wherein the communication interface communicates with the camera in a wireless manner.

4. The robot of claim 1 , wherein the control circuit is configured to:
control the robot arm to grasp the external object such that the center of the end effector coincides with the center of the external object, if the external object is placed at a specified position on the worktable.

5. The robot of claim 1 , further comprising:
an input device electrically connected with the control circuit to receive an input from a user of the robot,
wherein the control circuit is configured to control the robot arm to place the external object on the worktable if an instruction of the user of the robot is input through the input device, after the external object is grasped by the end effector such that the center of the end effector coincides with the center of the external object.

6. The robot of claim 1 , wherein the control circuit is configured to:
acquire the coordinates of the central point of the external object in the coordinate system of the camera by analyzing the image of the external object using a pattern of the external object.

7. The robot of claim 1 , wherein the control circuit is configured to:
control the robot arm to place the external object at a first position on the worktable after the external object is grasped by the end effector such that the center of the end effector coincides with the center of the external object;
acquire first coordinates of the end effector in the base coordinate system of the robot when the external object is placed at the first position;
acquire first coordinates of the central point of the external object in the coordinate system of the camera from a first image of the external object captured by the camera;
control the robot arm to grasp the external object placed at the first position and to place the external object at a second position different from the first position on the worktable;
acquire second coordinates of the end effector in the base coordinate system of the robot when the external object is placed at the second position;
acquire second coordinates of the central point of the external object in the coordinate system of the camera from a second image of the external object captured by the camera; and
calculate the calibration parameter for defining the relation between the coordinate system of the end effector and the coordinate system of the camera, based on the first coordinates of the end effector, the first coordinates of the central point of the external object, the second coordinates of the end effector, and the second coordinates of the central point of the external object.

8. The robot of claim 1 , wherein the control circuit is configured to:
calculate the calibration parameter based on a first parameter for defining a relation between the base coordinate system of the robot and the coordinate system of the camera and a second parameter for defining a relation between the base coordinate system of the robot and the coordinate system of the end effector.

9. The robot of claim 1 , wherein the calibration parameter includes a rotation matrix and a translation vector for defining the relation between the coordinate system of the end effector and the coordinate system of the camera.

10. The robot of claim 1 , wherein the calibration parameter includes a homography matrix associated with projective transformation.

11. The robot of claim 1 , wherein the calibration parameter includes a camera intrinsic parameter including at least one of a focal length or a lens distortion coefficient of the camera.

12. An electronic device for performing hand-eye calibration of a robot, the robot including a robot arm, the robot arm including an end effector, the robot having a base coordinate system, and the end effector having a coordinate system, the electronic device comprising:
at least one communication interface connected in a wireless or wired manner with the robot and a camera device, the camera device having a coordinate system and positioned outside the robot arm; and
at least one processor electrically connected with the at least one communication interface,
wherein the at least one processor is configured to:
control the robot to place an external object on a worktable after the external object is grasped by the end effector such that a center of the end effector coincides with a center of the external object;
control the robot to move out of sight of the camera without grasping the external object placed on the worktable, before an image of the external object is captured by the camera device,
acquire coordinates of a central point of the external object in the coordinate system of the camera device from an image of the external object captured by the camera device; and
calculate a calibration parameter for defining a relation between the coordinate system of the end effector and the coordinate system of the camera device, based on coordinates of the end effector in the base coordinate system of the robot and coordinates of the central point of the external object in the coordinate system of the camera device, when the external object is placed on the worktable.

13. The electronic device of claim 12 , wherein the at least one processor is further configured to:
acquire the coordinates of the central point of the external object by analyzing the image of the external object using a pattern of the external object.

14. The electronic device of claim 12 , wherein the at least one processor is further configured to:
control the robot to place the external object at a first position on the worktable after the external object is grasped by the end effector such that the center of the end effector coincides with

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Soon Yong
• Roh Kyung Shik
• Ahn Sung Hwan
• Yoon Suk June
• Lee So Hee
• Jeon Young Pil

Classifications

• B25J19/023

  including video camera means · CPC title

• B25J15/0004

  with provision for adjusting the gripped object in the hand · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• B25J9/1692 Primary

  Calibration of manipulator · CPC title

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title