Visual perception device based spatial calibration method and apparatus for robot body coordinate system, and storage medium

Patent metadata

Field	Value
Publication number	US-12202152-B2
Application number	US-202217734629-A
Country	US
Kind code	B2
Filing date	May 2, 2022
Priority date	Apr 28, 2020
Publication date	Jan 21, 2025
Grant date	Jan 21, 2025

Abstract

This disclosure relates to a spatial calibration method and apparatus of a robot ontology coordinate system based on a visual perception device and a storage medium. The method includes: obtaining first transformation relationships; obtaining second transformation relationships; using a transformation relationship between a visual perception coordinate system and an ontology coordinate system as an unknown variable; and resolving the unknown variable based on an equivalence relationship between a transformation relationship obtained according to the first transformation relationships and the unknown variable and a transformation relationship obtained according to the second transformation relationships and the unknown variable, to obtain the transformation relationship between the visual perception coordinate system and the ontology coordinate system.

First claim

What is claimed is:

1. A spatial calibration method of a robot ontology coordinate system based on a visual perception device, performed by a computer device, the method comprising:
obtaining first transformation relationships, the first transformation relationships including:
a transformation relationship between a calibration object coordinate system and a visual perception coordinate system when a target motion mechanism of a robot is located at a first sampling point, and
a transformation relationship between a motion mechanism coordinate system and an ontology coordinate system when the target motion mechanism of the robot is located at the first sampling point;

obtaining second transformation relationships, the second transformation relationships including:
a transformation relationship between the calibration object coordinate system and the visual perception coordinate system when the target motion mechanism is located at a second sampling point, and
a transformation relationship between the motion mechanism coordinate system and the ontology coordinate system when the target motion mechanism is located at the second sampling point; and

deriving a target transformation relationship based on an equivalence relationship between a transformation relationship obtained according to the first transformation relationships and the target transformation relationship and a transformation relationship obtained according to the second transformation relationships and the target transformation relationship, the target transformation relationship being a transformation relationship between the visual perception coordinate system and the ontology coordinate system, and a target calibration object corresponding to the calibration object coordinate system being disposed on the target motion mechanism.

2. The method according to claim 1 , wherein deriving the target transformation relationship comprises:
obtaining a first calculation result based on the transformation relationships between the calibration object coordinate system and the visual perception coordinate system when the target motion mechanism is located respectively at the first sampling point and the second sampling point;
obtaining a second calculation result based on the transformation relationships between the motion mechanism coordinate system and the ontology coordinate system when the target motion mechanism is located respectively at the first sampling point and the second sampling point; and
deriving the target transformation relationship based on an equivalence relationship between a first value calculated according to the first calculation result and the target transformation relationship and a second value calculated according to the second calculation result and the target transformation relationship.

3. The method according to claim 2 , wherein deriving the target transformation relationship based on the first value and the second value comprises:
obtaining a current solution corresponding to the target transformation relationship;
obtaining a first current value through calculation according to the current solution and the first calculation result;
obtaining a second current value through calculation according to the current solution and the second calculation result;
obtaining a difference calculation result according to a difference between the first current value and the second current value and adjusting the current solution in a direction of making the difference calculation result smaller to obtain an updated current solution; and
repeating to the operation of obtaining the first current value, obtaining the second current value, and adjusting the current solution until the difference calculation result is less than a first threshold, which current solution is used as the transformation relationship between the visual perception coordinate system and the ontology coordinate system.

4. The method according to claim 2 , wherein obtaining the second calculation result comprises:
obtaining a transformation relationship from the motion mechanism coordinate system to the ontology coordinate system when the target motion mechanism is located at the first sampling point, the transformation relationship being a first known value;
obtaining a transformation relationship from the ontology coordinate system to the motion mechanism coordinate system when the target motion mechanism is located at the second sampling point, and the transformation relationship being a second known value; and
multiplying the first known value by the second known value to obtain the second calculation result.

5. The method according to claim 1 , wherein obtaining the first transformation relationships comprises:
obtaining a projection matrix corresponding to a visual perception device; and
performing calculation according to the projection matrix and a transformation relationship between the calibration object coordinate system and an image coordinate system when the target motion mechanism is located at the first sampling point to obtain the transformation relationship between the calibration object coordinate system and the visual perception coordinate system when the target motion mechanism is located at the first sampling point.

6. The method according to claim 5 , wherein obtaining the first transformation relationships further comprises:
moving the target motion mechanism to the first sampling point and controlling the visual perception device to perform image acquisition on the target calibration object to obtain a target image; and
obtaining, according to coordinates of the target calibration object in the target image and a size parameter corresponding to the target calibration object, the transformation relationship between the calibration object coordinate system and the image coordinate system when the target motion mechanism is located at the first sampling point.

7. The method according to claim 1 , wherein obtaining the first transformation relationships comprises:
obtaining a transformation relationship between coordinate systems of adjacent joints in a joint sequence corresponding to the target motion mechanism when the target motion mechanism is located at the first sampling point to obtain an adjacent transformation relationship set; and
performing multiplication on transformation relationships in the adjacent transformation relationship set according to an arrangement sequence of joints in the joint sequence to obtain the transformation relationship between the motion mechanism coordinate system and the ontology coordinate system when the target motion mechanism is located at the first sampling point.

8. A robot control method, performed by a computer device, the method comprising:
obtaining visual pose information of a target object in a visual perception coordinate system corresponding to a robot;
obtaining a target transformation relationship between the visual perception coordinate system and an ontology coordinate system of the robot, the transformation relationship between the visual perception coordinate system and the ontology coordinate system being resolved according to an equivalence relationship between transformation relationships obtained through calculation respectively according to first transformation relationships and second transformation relationships;
obtaining target pose information corresponding to the target object according to the visual pose information and the transformation relationship between the visual perception coordinate system and the ontology coordinate system; and
controlling, according to the target pose information, a target motion mechanism of the robot to move,
the first transformation relationships and the second transformati

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Chi Wanchao
• Chen Le
• Ling Yonggen
• Zhang Shenghao
• Zheng Yu
• Jiang Xinyang
• Zhang Zhengyou

Classifications

• B62D57/032

  with alternately or sequentially lifted supporting base and legs; with alternately or sequentially lifted feet or skid (B62D57/024 takes precedence) · CPC title

• B25J19/023

  including video camera means · CPC title

• B25J9/1692

  Calibration of manipulator · CPC title

• B25J9/1602

  characterised by the control system, structure, architecture · CPC title

• G05B2219/37009

  Calibration of vision system, camera, adapt light level · CPC title