Object pose tracking method and apparatus

Patent metadata

Field	Value
Publication number	US-2020184682-A1
Application number	US-201916548868-A
Country	US
Kind code	A1
Filing date	Aug 23, 2019
Priority date	Dec 11, 2018
Publication date	Jun 11, 2020
Grant date	—

Abstract

The present disclosure relates to robot technology, which provides an object pose tracking method and apparatus. The method includes: obtaining an initial pose of the object, and determining a first viewing angle of an object with respect to a camera based on the initial pose; searching for a first 3D model corresponding to the first viewing angle; calculating a projected contour of an initial pose using the searched first 3D model, and obtaining feature point(s) of the projected contour; calculating a pose change amount based on the feature point(s) of the projected contour and feature point(s) of a real contour; and performing an iterative calculation on the pose change amount to obtain a pose correction amount, and obtaining a corrected pose of the object based on the pose correction amount. In the calculation process, the 3D model can be selected to improve the calculation speed by fewer feature points.

First claim

What is claimed is:

1 . A computer-implemented object pose tracking method, comprising executing on a processor steps of:
obtaining an initial pose of the object, and determining a first viewing angle of an object with respect to a camera based on the initial pose; searching for a first 3D model corresponding to the first viewing angle based on a correspondence between 3D models and viewing angles; calculating a projected contour of an initial pose using the searched first 3D model, and obtaining one or more feature points of the projected contour; calculating a pose change amount based on the one or more feature points of the projected contour and one or more feature points of a real contour of an image captured through the camera; and performing an iterative calculation on the pose change amount to obtain a pose correction amount, and obtaining a corrected pose of the object based on the pose correction amount.

2 . The method of claim 1 , wherein after the step of obtaining the corrected pose of the object based on the pose correction amount further comprises:
calculating a pose of the object corresponding to a next frame by taking the corrected pose of the object as the initial pose of the next frame.

3 . The method of claim 2 , wherein the step of calculating the pose of the object corresponding to the next frame by taking the corrected pose of the object as the initial pose of the next frame comprises:
determining a second viewing angle of the object with respect to the camera based on the initial pose of the next frame; searching for a second 3D model corresponding to the second viewing angle based on the pre-stored correspondence between 3D models and viewing angles; and correcting one or more pose parameters based on a statistical value, in response to the first 3D model and the second 3D model having changed.

4 . The method of claim 1 , wherein the step of calculating the projected contour of the initial pose using the searched first 3D model comprises:
projecting the object from a world coordinate system to a camera coordinate system based on the searched first 3D model and the initial pose to obtain an image of the camera coordinate system; and converting the image of the camera coordinate system into an image of an image coordinate system based on one or more internal parameters of the camera, and calculating the projected contour based on the image of the image coordinate system.

5 . The method of claim 1 , wherein the step of calculating the pose change amount based on the one or more feature points of the projected contour and the one or more feature points of the real contour of the image captured through the camera comprises:
determining the one or more feature points of the real contour of a current frame of the image captured through the camera based on a change in a RGB color intensity of the captured image; calculating a change deviation of each of the feature points based on the one or more feature points of the real contour of the current frame and the one or more feature points of the projected contour; and calculating the pose change amount corresponding to the change deviation using a linear equation.

6 . The method of claim 1 , wherein the step of obtaining the initial pose of the object comprises:
collecting, through the camera, an image with the object; performing a feature extraction on the image with the object to obtain one or more features; and performing a pose estimation based on the one or more feature(s) to obtain the initial pose.

7 . The method of claim 1 , wherein the correspondence comprises ranges of viewing angles each corresponds to a 3D model.

8 . An object pose tracking apparatus, comprising:
a camera; a memory; a processor, and one or more computer programs stored in the memory and executable on the processor, wherein the one or more computer programs comprise: a viewing angle determining unit configured to obtain an initial pose of the object, and determine a first viewing angle of an object with respect to a camera based on the initial pose; a model searching unit configured to search for a first 3D model corresponding to the first viewing angle based on a pre-stored correspondence between 3D models and viewing angles; a feature point obtaining unit configured to calculate a projected contour of an initial pose using the searched first 3D model, and obtain one or more feature points of the projected contour, a pose change amount calculating unit configured to calculate a pose change amount based on the one or more feature points of the projected contour and one or more feature points of a real contour of an image captured through the camera; and a pose correcting unit configured to perform an iterative calculation on the pose change amount to obtain a pose correction amount, and obtain a corrected pose of the object based on the pose correction amount.

9 . The apparatus of claim 8 , wherein the one or more computer programs further comprise:
a next frame calculating unit configured to calculate a pose of the object corresponding to a next frame by taking the corrected pose of the object as the initial pose of the next frame.

10 . The apparatus of claim 9 , wherein the next frame calculating unit is configured to:
determine a second viewing angle of the object with respect to the camera based on the initial pose of the next frame; search for a second 3D model corresponding to the second viewing angle based on the pre-stored correspondence between 3D models and viewing angles; and correct one or more pose parameters based on a statistical value, in response to the first 3D model and the second 3D model having changed.

11 . The apparatus of claim 8 , wherein the feature point obtaining unit is configured to:
project the object from a world coordinate system to a camera coordinate system based on the searched first 3D model and the initial pose to obtain an image of the camera coordinate system; and convert the image of the camera coordinate system into an image of an image coordinate system based on one or more internal parameters of the camera, and calculate the projected contour based on the image of the image coordinate system.

12 . The apparatus of claim 8 , wherein the pose change amount calculating unit is configured to:
determine the one or more feature points of the real contour of a current frame of the image captured through the camera based on a change in a RGB color intensity of the captured image; calculate a change deviation of each of the feature points based on the one or more feature points of the real contour of the current frame and the one or more feature points of the projected contour; and calculate the pose change amount corresponding to the change deviation using a linear equation.

13 . The apparatus of claim 8 , wherein the viewing angle determining unit is configured to:
collect, through the camera, an image with the object; perform a feature extraction on the image with the object to obtain one or more features; and perform a pose estimation based on the one or more feature(s) to obtain the initial pose.

14 . The apparatus of claim 8 , wherein the correspondence comprises ranges of viewing angles each corresponds to a 3D model.

Assignees

• Ubtech Robotics Corp Ltd

Inventors

• Xiong Youjun
• Zhang Wanli
• Pang Jianxin

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T7/246 Primary

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06T7/251

  involving models · CPC title