Mobile robot control method, computer-implemented storage medium and mobile robot

Patent metadata

Field	Value
Publication number	US-11429112-B2
Application number	US-202017138946-A
Country	US
Kind code	B2
Filing date	Dec 31, 2020
Priority date	Dec 31, 2020
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

A mobile robot control method includes: acquiring a first image that is captured by a camera on a robot when the robot is in a desired pose; acquiring a second image that is captured by the camera on the robot when the robot is in a current pose; extracting multiple pairs of matching feature points from the first image and the second image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points, wherein a center of the virtual unitary sphere is coincident with an optical center of coordinates of the camera; acquiring an invariant image feature and a rotation vector feature based on the multiple projection feature points, and controlling the robot to move until the robot is in the desired pose according to the invariant image feature and the rotation vector feature.

First claim

What is claimed is:

1. A computer-implemented method executed by one or more processors for controlling a mobile robot, the method comprising:
acquiring a first image that is captured by a camera on the robot when the robot is in a desired pose;
acquiring a second image that is captured by the camera on the robot when the robot is in a current pose;
extracting multiple pairs of matching feature points from the first image and the second image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points, wherein a center of the virtual unitary sphere is coincident with an optical center of coordinates of the camera;
acquiring an invariant image feature and a rotation vector feature based on the multiple projection feature points, and controlling the robot to move until the robot is in the desired pose according to the invariant image feature and the rotation vector feature.

2. The method of claim 1 , wherein controlling the robot to move comprises:
controlling the robot to translate according to the invariant image feature; and
controlling the robot to rotate according to the rotation vector feature.

3. The method of claim 2 , wherein controlling the robot to translate according to the invariant image feature, comprises:
calculating a first angular velocity and a first linear velocity based on the invariant image feature and a preset control model;
controlling the robot to translate according to the first angular velocity and the first linear velocity;
determining whether a translational error is less than a first preset threshold;
going back to a step of acquiring the second image if the translational error is greater than or equal to the first preset threshold; and
controlling the robot to rotate according to the rotation vector feature if the translational error is less than the first preset threshold.

4. The method of claim 2 , wherein controlling the robot to rotate according to the rotation vector feature, comprises:
calculating a second angular velocity and a second linear velocity based on the rotation vector feature and the control model;
controlling the robot to rotate according to the second angular velocity and the second linear velocity;
determining whether an orientation error of the robot after rotation of the robot is less than a second preset threshold;
acquiring a third image by the camera if the orientation error of the robot is greater than or equal to the second preset threshold;
extracting multiple pairs of matching feature points from the first image and the third image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points; and
acquiring the rotation vector feature based on the multiple projection feature points that are extracted from the first image and the third image, and then going back to a step of calculating the second angular velocity and the second linear velocity based on the rotation vector feature and the control model.

5. The method of claim 1 , wherein the invariant image feature comprises one or more of a reciprocal of a distance between two of the projection feature points, an image moment, and an area.

6. The method of claim 1 , wherein acquiring the invariant image feature based on the multiple projection feature points, comprises:
acquiring at least two of a distance between two of the projection feature points, an image moment, and an area; and
calculating an average value of the at least two of the distance between two of the projection feature points, the image moment, and the area, and using the average value as the invariant image feature.

7. The method of claim 1 , wherein acquiring the rotation vector feature based on the multiple projection feature points, comprises:
determining an acceleration direction of the robot based on the multiple projection feature points;
using an angle between the acceleration direction and an x-axis of a robot coordinate system as the rotation vector feature.

8. The method of claim 1 , wherein extracting multiple pairs of matching feature points from the first image and the second image, comprises:
extracting a first number of original feature points from each of the first image and the second image using scale-invariant feature transform; and
acquiring a second number of pairs of matching feature points by comparing and matching the extracted original feature points.

9. The method of claim 1 , further comprising:
control the robot to stop at a desired position and in a preset pose; and
using an image of an environment in front of the robot captured by the camera as the first image, wherein the environment in front of the robot comprises at least three feature points.

10. A non-transitory computer-readable storage medium storing one or more programs to be executed in a mobile robot, the one or more programs, when being executed by one or more processors of the robot, causing the robot to perform processing comprising:
acquiring a first image that is captured by a camera on the robot when the robot is in a desired pose;
acquiring a second image that is captured by the camera on the robot when the robot is in a current pose;
extracting multiple pairs of matching feature points from the first image and the second image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points, wherein a center of the virtual unitary sphere is coincident with an optical center of coordinates of the camera;
acquiring an invariant image feature and a rotation vector feature based on the multiple projection feature points; and controlling the robot to move until the robot is in the desired pose according to the invariant image feature and the rotation vector feature.

11. The non-transitory computer-readable storage medium of claim 10 , wherein the invariant image feature comprises one or more of a reciprocal of a distance between two of the projection feature points, an image moment, and an area;
acquiring the rotation vector feature based on the multiple projection feature points comprises:
determining an acceleration direction of the robot based on the multiple projection feature points;
using an angle between the acceleration direction and an x-axis of a robot coordinate system as the rotation vector feature.

12. A mobile robot comprising:
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprises:
instructions for acquiring a first image that is captured by a camera on the robot when the robot is in a desired pose;
instructions for acquiring a second image that is captured by the camera on the robot when the robot is in a current pose;
instructions for extracting multiple pairs of matching feature points from the first image and the second image, and projecting the extracted feature points onto a virtual unitary sphere to obtain multiple projection feature points, wherein a center of the virtual unitary sphere is coincident with an optical center of coordinates of the camera;
instructions for acquiring an invariant image feature and a rotation vector feature based on the multiple projection feature points, and instructions for controlling the robot to move until the robot is in the desired pose according to the invariant image feature and the rotation vector feature.

13. The robot of claim 12 , wherein controlling the robot to move comprises:
controlling the robot to translate according to the invariant imag

Assignees

• Ubtech North America Res And Development Center Corp
• Ubtech Robotics Corp Ltd

Inventors

• Guo Dejun
• Shao Dan
• Shen Yang
• Peng Kang-Hao
• Tan Huan

Classifications

• G05D1/0223

  involving speed control of the vehicle (vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00) · CPC title

• G05D1/0253 Primary

  extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70) · CPC title