Simultaneous kinematic and hand-eye calibration

Patent metadata

Field	Value
Publication number	US-10076842-B2
Application number	US-201615278156-A
Country	US
Kind code	B2
Filing date	Sep 28, 2016
Priority date	Sep 28, 2016
Publication date	Sep 18, 2018
Grant date	Sep 18, 2018

Abstract

Described are machine vision systems and methods for simultaneous kinematic and hand-eye calibration. A machine vision system includes a robot or motion stage and a camera in communication with a control system. The control system is configured to move the robot or motion stage to poses, and for each pose: capture an image of calibration target features and robot joint angles or motion stage encoder counts. The control system is configured to obtain initial values for robot or motion stage calibration parameters, and determine initial values for hand-eye calibration parameters based on the initial values for the robot or motion stage calibration parameters, the image, and joint angles or encoder counts. The control system is configured to determine final values for the hand-eye calibration parameters and robot or motion stage calibration parameters by refining the hand-eye calibration parameters and robot or motion stage calibration parameters to minimize a cost function.

First claim

What is claimed is:

1. A machine vision system comprising:
a robot;
a camera; and
a control system in communication with the robot and the camera, the control system configured to:
move the robot to a plurality of poses, and for each pose of the plurality of poses:
capture, with the camera, an image of one or more features on a calibration target; and
capture a plurality of robot joint angles;

obtain a plurality of initial values for a plurality of robot calibration parameters;
determine a plurality of initial values for a plurality of hand-eye calibration parameters based on the plurality of initial values for the plurality of robot calibration parameters, and the image and the plurality of robot joint angles captured for each of the plurality of poses;
determine a plurality of final values for the plurality of hand-eye calibration parameters and a plurality of final values for the robot calibration parameters by refining one or more of the plurality of hand-eye calibration parameters and one or more of the plurality of robot calibration parameters to minimize a cost function.

2. The machine vision system of claim 1 , wherein the control system is further configured to:
use the plurality of final values for the plurality of hand-eye calibration parameters and the plurality of final values for the robot calibration parameters for a vision guided robot application.

3. The machine vision system of claim 1 , wherein the control system is further configured to:
move the robot using the plurality of final values for the robot calibration parameters.

4. The machine vision system of claim 1 , wherein the cost function measures, for each pose of the plurality of poses, a difference between an observed location of a feature in the image captured for the pose and an expected location of the feature, the expected location determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, and the plurality of robot joint angles captured for the pose.

5. The machine vision system of claim 1 , wherein the cost function measures, for each pose of the plurality of poses, a difference between a composite transformation and an identity transformation, the composite transformation being a composition of a first transformation from a first coordinate space to a second coordinate space and a second transformation from the second coordinate space to the first coordinate space, the first transformation and the second transformation determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, the plurality of observed features of the calibration target for the pose, and the plurality of robot joint angles captured for the pose.

6. The machine vision system of claim 1 , wherein the cost function measures, for each pose of the plurality of poses, a difference between a robot joint angle of the plurality of robot joint angles captured for the pose and an estimated joint angle for the pose, the estimated joint angle determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, and the plurality of observed features of the calibration target for the pose.

7. The machine vision system of claim 1 , wherein the control system is further configured to minimize the cost function using non-linear least squares.

8. The machine vision system of claim 1 , wherein the control system is configured to refine less than all of the plurality of hand-eye calibration parameters and less than all of the plurality of robot calibration parameters.

9. The machine vision system of claim 1 , wherein the plurality of robot calibration parameters comprises Denavit-Hartenberg parameters for the robot.

10. A machine vision system-implemented calibration method comprising:
moving, by a control system, a robot to a plurality of poses, and for each pose of the plurality of poses:
capturing, by a camera, an image of one or more features on a calibration target; and
capturing, by the control system, a plurality of robot joint angles;

obtaining, by the control system, a plurality of initial values for a plurality of robot calibration parameters;
determining, by the control system, a plurality of initial values for a plurality of hand-eye calibration parameters based on the plurality of initial values for the plurality of robot calibration parameters, and the image and the plurality of robot joint angles captured for each of the plurality of poses;
determining, by the control system, a plurality of final values for the plurality of hand-eye calibration parameters and a plurality of final values for the plurality of robot calibration parameters by refining one or more of the plurality of hand-eye calibration parameters and one or more of the plurality of robot calibration parameters to minimize a cost function.

11. The method of claim 10 , wherein the cost function measures, for each pose of the plurality of poses, a difference between an observed location of a feature in the image captured for the pose and an expected location of the feature, the expected location determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, and the plurality of robot joint angles captured for the pose.

12. The method of claim 10 , wherein the cost function measures, for each pose of the plurality of poses, a difference between a composite transformation and an identity transformation, the composite transformation being a composition of a first transformation from a first coordinate space to a second coordinate space and a second transformation from the second coordinate space to the first coordinate space, the first transformation and the second transformation determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, the plurality of observed features of the calibration target for the pose, and the plurality of robot joint angles captured for the pose.

13. The method of claim 10 , wherein the cost function measures, for each pose of the plurality of poses, a difference between a robot joint angle of the plurality of robot joint angles captured for the pose and an estimated joint angle for the pose, the estimated joint angle determined based on a first plurality of values for the plurality of robot calibration parameters, a second plurality of values for the plurality of hand-eye calibration parameters, and the plurality of observed features of the calibration target for the pose.

14. The method of claim 10 , further comprising minimizing the cost function using non-linear least squares.

15. The method of claim 10 , further comprising refining less than all of the plurality of hand-eye calibration parameters and less than all of the plurality of robot calibration parameters.

16. The method of claim 10 , wherein the plurality of robot calibration parameters comprises Denavit-Hartenberg parameters for the robot.

17. A machine vision system comprising:
a motion stage;
a camera; and
a control system in communication with the motion stage and the camera, the control system configured to:
move the motion stage to a plurality of poses, and for each pose of t

Assignees

• Cognex Corp

Inventors

• Liu Lifeng
• Marrion Cyril C
• Gan Tian

Classifications

• G06T7/74

  involving reference images or patches · CPC title

• G05B2219/40607

  Fixed camera to observe workspace, object, workpiece, global · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G05B2219/39045

  Camera on end effector detects reference pattern · CPC title

• B25J19/04

  Viewing devices · CPC title