Method and apparatus for managing robot system

Patent metadata

Field	Value
Publication number	US-11577400-B2
Application number	US-202017034162-A
Country	US
Kind code	B2
Filing date	Sep 28, 2020
Priority date	Sep 3, 2018
Publication date	Feb 14, 2023
Grant date	Feb 14, 2023

Abstract

Embodiments of the present disclosure provide methods for managing a robot system. In one method, orientations for links in the robot system may be obtained when the links are arranged in at least one posture, here each of the orientations indicates a direction pointed by one of the links. At least one image of an object placed in the robot system may be obtained from a vision device equipped on one of the links. Based on the orientations and the at least one image, a first mapping may be determined between a vision coordinate system of the vision device and a link coordination system of the link. Further, embodiments of present disclosure provide apparatuses, systems, and computer readable media for managing a robot system. The vision device may be calibrated by the first mapping and may be used to manage operations of the robot system.

First claim

What is claimed is:

1. A method for managing a robot system, comprising:
obtaining orientations for links in the robot system when the links are arranged in a first posture, each of the orientations indicating a direction pointed by one of the links;
obtaining, from a vision device equipped on one of the links, at least one image of an object placed in the robot system; and
determining, based on the orientations and the at least one image, a first mapping between a vision coordinate system of the vision device and a link coordinate system of the link;
wherein the first posture includes any posture in the robot system;
wherein the first posture is not limited to a fixed position.

2. The method of claim 1 , wherein determining the first mapping comprises:
determining, based on the orientations, a second mapping between the link coordinate system and a world coordinate system of the robot system;
determining, based on the at least one image, a third mapping between the vision coordinate system and the world coordinate system; and
determining the first mapping based on a transformation relationship between the second and third mappings.

3. The method of claim 2 , wherein determining the second mapping further comprises:
determining the second mapping based on positions of the links;
wherein the positions of the links include a height and a width of the link.

4. The method of claim 2 , wherein determining the third mapping comprises:
obtaining a first measurement and a second measurement for a first point and a second point in an image of the at least one image respectively, the first and second points located in a first axis of the vision coordinate system;
obtaining a third measurement of a third point in the image, the third point located in a second axis of the vision coordinate system; and
determining the third mapping based on the obtained first, second and third measurements.

5. The method of claim 1 , further comprising:
calibrating the vision device with a calibration board of the vision device before the obtaining the at least one image.

6. The method of claim 1 , wherein, obtaining the orientations for the links comprises:
obtaining a first group of orientations for the links when the group of links are arranged in the first posture;
obtaining a second group of orientations for the links when the group of links are arranged in a second posture; and
obtaining the at least one image of the object comprises:
obtaining a first image when the links are arranged in the first posture, and
obtaining a second image when the links are arranged in the second posture.

7. The method of claim 6 , wherein determining the first mapping comprises:
constructing a transformation relationship between the first mapping, the first and second groups of orientations and the first and second images, the first mapping being an unknown variable in the transformation relationship; and
solving the transformation relationship so as to determine the first mapping.

8. The method of claim 1 , further comprising:
obtaining, from the vision device, an image of a target object to be processed by the robot system;
determining a source coordinate of the target object in the obtained image, the source coordinate represented in the vision coordinate system;
determining a destination coordinate of the target object based on the source coordinate and the first mappings, the destination coordinate represented in a world coordinate system; and
processing the target object based on the destination coordinate.

9. An apparatus for managing a robot system, comprising:
an orientation obtaining unit configured to obtain orientations for links in the robot system when the links are arranged in a first posture, each of the orientations indicating a direction pointed by one of the links;
an image obtaining unit configured to obtain, from a vision device equipped on one of the links, at least one image of an object placed in the robot system; and
a determining unit configured to determine, based on the orientations and the at least one image, a first mapping between a vision coordinate system of the vision device and a link coordination system of the link;
wherein the first posture includes any posture in the robot system;
wherein the first posture is not limited to a fixed position.

10. The apparatus of claim 9 , wherein the determining unit comprises:
a second mapping determining unit configured to determine, based on the orientations, a second mapping between the link coordinate system and a world coordinate system of the robot system;
a third mapping determining unit configured to determine, based on the at least one image, a third mapping between the vision coordinate system and the world coordinate system; and
a first mapping determining unit configured to determine the first mapping based on a transformation relationship between the second and third mappings.

11. The apparatus of claim 10 , wherein the second mapping determining unit is further configured to:
determine the second mapping based on positions of the links;
wherein the positions of the links are based on a height and width of the link.

12. The apparatus of claim 10 , wherein the third mapping determining unit comprises:
a measurement obtaining unit configured to:
obtain a first measurement and a second measurement for a first point and a second point in an image of the at least one image respectively, the first and second points located in a first axis of the vision coordinate system;
obtain a third measurement of a third point in the image, the third point located in a second axis of the vision coordinate system; and

a mapping determining unit configured to determine the third mapping based on the obtained first, second and third measurements.

13. The apparatus of claim 9 , further comprising:
a calibrating unit configured to calibrate the vision device with a calibration board of the vision device before the obtaining the at least one image.

14. The apparatus of claim 9 , wherein,
the orientation of obtaining unit is further configured to:
obtain a first group of orientations for the links when the group of links are arranged in the first posture, and
obtain a second group of orientations for the links when the group of links are arranged in a second posture; and

the image obtaining unit is further configured to:
obtain a first image when the links are arranged in the first posture, and
obtain a second image when the links are arranged in the second posture.

15. The apparatus of claim 14 , wherein the determining unit comprises:
a constructing unit configured to construct a transformation relationship between the first mapping, the first and second groups of orientations and the first and second images, the first mapping being an unknown variable in the transformation relationship; and
a solving unit configured to solve the transformation relationship so as to determine the first mapping.

16. The apparatus of claim 9 , wherein,
the image obtaining unit further configured to obtain, from the vision device, an image of a target object to be processed by the robot system; and
further comprising:
a source determining unit configured to determine a source coordinate of the target object in the obtained image, the source coordinate represented in the vision coordinate system;
a destination determining unit configured to determine a destination coordinate of the target object based on the source coordinate and the first mappings, the destination coordinate represented in a world coordinate

Assignees

• Abb Schweiz Ag

Inventors

• Qian Changnian
• Wang Yanjun
• Ma Bojun

Classifications

• B25J9/106

  with articulated links · CPC title

• B25J13/089

  Determining the position of the robot with reference to its environment · CPC title

• G05B2219/39057

  Hand eye calibration, eye, camera on hand, end effector · CPC title

• B25J9/1697 Primary

  Vision controlled systems · CPC title

• G05B2219/39045

  Camera on end effector detects reference pattern · CPC title