Method and apparatus for three-dimensional object pose estimation, device and storage medium

Patent metadata

Field	Value
Publication number	US-11145080-B2
Application number	US-201916508372-A
Country	US
Kind code	B2
Filing date	Jul 11, 2019
Priority date	Aug 3, 2018
Publication date	Oct 12, 2021
Grant date	Oct 12, 2021

Abstract

The present application provides a method and an apparatus for three-dimensional object pose estimation, a device and a storage medium. The method includes: calculating a graph of a previous frame and a graph of a current frame for a target three-dimensional object; performing a matching calculation on the graph of the previous frame and the graph of the current frame using a graph matching algorithm to obtain a vertex correspondence relationship between the graph of the previous frame and the graph of the current frame; calculating a pose of the target three-dimensional object in the current frame according to the vertex correspondence relationship, a pose of the target three-dimensional object in the previous frame and a PnP algorithm. The matching accuracy of feature points is effectively improved, and thereby the accuracy of three-dimensional object pose estimation is improved.

First claim

What is claimed is:

1. A method for three-dimensional object pose estimation, comprising:
calculating a graph of a previous frame and a graph of a current frame for a target three-dimensional object;
performing a matching calculation on the graph of the previous frame and the graph of the current frame by using a graph matching algorithm to obtain a vertex correspondence relationship between the graph of the previous frame and the graph of the current frame;
calculating a pose of the target three-dimensional object in the current frame according to the vertex correspondence relationship, a pose of the target three-dimensional object in the previous frame and a perspective n-point (PnP) algorithm;
wherein the calculating a graph of a previous frame and a graph of a current frame for the target three-dimensional object, comprises:
obtaining a mask image of the previous frame for the target three-dimensional object;
extracting feature points of the target three-dimensional object in an image of the previous frame in a region with a pixel value of 1 in the mask image of the previous frame, and extracting the feature points of the target three-dimensional object in an image of the current frame in the region with the pixel value of 1 in the mask image of the previous frame;
connecting adjacent feature points corresponding to the image of the previous frame to form the graph of the previous frame;
connecting adjacent feature points corresponding to the image of the current frame to form the graph of the current frame;
wherein vertices in the graph of the previous frame and the graph of the current frame are the feature points, and a weight of an edge is an average value of response values of two feature points corresponding to the edge.

2. The method of claim 1 , wherein the extracting feature points of the target three-dimensional object in an image of the previous frame in a region with a pixel value of 1 in the mask image of the previous frame, and extracting the feature points of the target three-dimensional object in an image of the current frame in the region with the pixel value of 1 in the mask image of the previous frame, comprises:
using a SIFT algorithm to extract the feature points of the target three-dimensional object in the previous frame image in the region with the pixel value of 1 in the mask image of the previous frame, and using the SIFT algorithm to extract the feature points of the target three-dimensional object in the current frame image in the region with the pixel value of 1 in the mask image of the previous frame.

3. The method of claim 1 , wherein the performing a matching calculation on the graph of the previous frame and the graph of the current frame using a graph matching algorithm to obtain a vertex correspondence relationship between the graph of the previous frame and the graph of the current frame, comprises:
inputting the graph of the previous frame and the graph of the current frame into a model of the graph matching algorithm to perform the matching calculation on the graph of the previous frame and the graph of the current frame;
outputting the vertex correspondence relationship between the graph of the previous frame and the graph of the current frame.

4. The method of claim 1 , wherein the calculating a pose of the target three-dimensional object in the current frame according to the vertex correspondence relationship, a pose of the target three-dimensional object in the previous frame and a PnP algorithm, comprises:
inputting the vertex correspondence relationship and the pose of the target three-dimensional object in the previous frame into a model of the PnP algorithm to calculate the pose of the target three-dimensional object in the current frame;
outputting the pose of the target three-dimensional object in the current frame.

5. A terminal device, comprising:
one or more processors;
a memory, configured to store one or more programs;
wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to:
calculate a graph of a previous frame and a graph of a current frame for a target three-dimensional object;
perform a matching calculation on the graph of the previous frame and the graph of the current frame by using a graph matching algorithm to obtain a vertex correspondence relationship between the graph of the previous frame and the graph of the current frame;
calculate a pose of the target three-dimensional object in the current frame according to the vertex correspondence relationship, a pose of the target three-dimensional object in the previous frame and a perspective n-point (PnP) algorithm;
wherein the one or more processors are further caused to:
obtain a mask image of the previous frame for the target three-dimensional object;
extract feature points of the target three-dimensional object in the image of the previous frame in a region with a pixel value of 1 in the mask image of the previous frame, and extract the feature points of the target three-dimensional object in the image of the current frame in the region with the pixel value of 1 in the mask image of the previous frame;
connect adjacent feature points corresponding to the image of the previous frame to form the graph of the previous frame, and connect adjacent feature points corresponding to the image of the current frame to form the graph of the current frame;
wherein vertices in the graph of the previous frame and the graph of the current frame are feature points, and a weight of an edge is an average value of response values of two feature points corresponding to the edge.

6. The terminal device of claim 5 , wherein the one or more processors are further caused to:
use a SIFT algorithm to extract the feature points of target three-dimensional object in the image of the previous frame in the region with the pixel value of 1 in the mask image of the previous frame, and use the SIFT algorithm to extract the feature points of the target three-dimensional object in the image of the current frame in the region with the pixel value of 1 in the mask image of the previous frame.

7. The terminal device of claim 5 , wherein the one or more processors are further caused to:
input the graph of the previous frame and the graph of the current frame into a model of the graph matching algorithm to perform the matching calculation on the graph of the previous frame and the graph of the current frame; output the vertex correspondence relationship between the graph of the previous frame and the graph of the current frame.

8. The terminal device of claim 5 , wherein the one or more processors are further caused to:
input the vertex correspondence relationship and the pose of the target three-dimensional object in the previous frame into a model of the PnP algorithm to calculate the pose of the target three-dimensional object in the current frame; output the pose of the target three-dimensional object in the current frame.

9. A non-transitory computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method of claim 1 .

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Chen Sili
• Lin Yuan
• Zhang Yongjie

Classifications

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title

• G06T7/75 Primary

  involving models · CPC title

• G06V10/84

  using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title