Method and apparatus for determining pose of image capturing device, and storage medium

Patent metadata

Field	Value
Publication number	US-11270460-B2
Application number	US-202016940148-A
Country	US
Kind code	B2
Filing date	Jul 27, 2020
Priority date	Apr 27, 2018
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

A method for determining a pose of an image capturing device is performed at an electronic device. The electronic device acquires a plurality of image frames captured by the image capturing device, extracts a plurality of matching feature points from the plurality of image frames and determines first position information of each of the matching feature points in each of the plurality of image frames. After estimating second position information of each of the matching feature points in a current image frame in the plurality of image frames by using the first position information of each of the matching feature points extracted from a previous image frame in the plurality of image frames, the electronic device determines a pose of the image capturing device based on the first position information and the second position information of each of the matching feature points in the current image frame.

First claim

What is claimed is:

1. A method for determining a pose of an image capturing device performed at an electronic device having a processor and memory storing a plurality of programs to be executed by the processor, the method comprising:
acquiring a plurality of image frames captured by the image capturing device;
extracting a plurality of matching feature points from the plurality of image frames and determining first position information of each of the matching feature points in each of the plurality of image frames;
estimating second position information of each of the matching feature points in a current image frame in the plurality of image frames by using the first position information of each of the matching feature points extracted from a previous image frame in the plurality of image frames; and
determining a pose of the image capturing device based on the first position information and the second position information of each of the matching feature points in the current image frame.

2. The method according to claim 1 , wherein the extracting a plurality of matching feature points in the plurality of image frames and determining first position information of each of the matching feature points in each of the plurality of image frames comprises:
acquiring two-dimensional position information of the first position information of each of the matching feature points in each image frame; and
transforming the two-dimensional position information of the first position information into three-dimensional position information of the first position information by using an internal parameter matrix of the image capturing device, the three-dimensional position information of the first position information comprising depth information of a scene.

3. The method according to claim 2 , wherein in a case that the current image frame is a first image frame in the plurality of image frames, the method further comprises:
before the estimating second position information of each of the matching feature points in a current image frame in the plurality of image frames by using the first position information of each of the matching feature points extracted from a previous image frame in the plurality of image frames:
initializing the three-dimensional position information of the first position information of each of the matching feature points in the current image frame; and
determining an initial pose of the image capturing device through homography matrix transformation based on initialization information of the three-dimensional position information of the first position information.

4. The method according to claim 2 , wherein the estimating second position information of each of the matching feature points in a current image frame in the plurality of image frames by using the first position information of each of the matching feature points extracted from a previous image frame in the plurality of image frames comprises:
converting two-dimensional position information of the first position information of each of the matching feature points in the previous image frame into two-dimensional position information of the second position information of each of the matching feature points in the current image frame based on the internal parameter matrix and a motion parameter matrix used in a process of acquiring the previous image frame to the current image frame by the image capturing device,
the motion parameter matrix comprising a rotation matrix and a displacement vector of the image capturing device, the rotation matrix representing posture information in the pose of the image capturing device, and the displacement vector representing position information in the pose of the image capturing device.

5. The method according to claim 4 , wherein the determining a pose of the image capturing device based on the first position information and the second position information of each of the matching feature points in the current image frame comprises:
calculating a difference between the two-dimensional position information of the first position information of each of the matching feature points in the current image frame and the two-dimensional position information of the second position information of each of the matching feature points in the current image frame;
estimating the motion parameter matrix and the three-dimensional position information of the first position information of each of the matching feature points in the current image frame by minimizing the difference; and
updating a pose of the image capturing device during capturing of the current image frame based on the rotation matrix and the displacement vector in the estimated motion parameter matrix.

6. The method according to claim 5 , wherein the two-dimensional position information of the second position information of each of the matching feature points in the current image frame is a projection of a two-dimensional position of the first position information of the matching feature point extracted from the previous image frame on the current image frame, and the estimating the motion parameter matrix and the three-dimensional position information of the first position information of each of the matching feature points in the current image frame by minimizing the difference comprises:
solving the motion parameter matrix and the three-dimensional position information of the first position information of the matching feature point in the current image frame in a case that a quadratic sum of a difference between the two-dimensional position information of the first position information of each of the matching feature points in the current image frame and the projection is minimized.

7. The method according to claim 5 , wherein the determining a pose of the image capturing device based on the first position information and the second position information of each of the matching feature points in the current image frame further comprises:
determining the pose as a final pose of the image capturing device in a case that a pose of the image capturing device determined according to a current matching feature point is the same as a pose of the image capturing device determined according to a previous matching feature point.

8. An electronic device, comprising:
a processor; and
a memory configured to store a plurality of executable instructions that, when executed by the processor, cause the electronic device to perform a plurality of operations including:
acquiring a plurality of image frames captured by an image capturing device;
extracting a plurality of matching feature points from the plurality of image frames and determining first position information of each of the matching feature points in each of the plurality of image frames;
estimating second position information of each of the matching feature points in a current image frame in the plurality of image frames by using the first position information of each of the matching feature points extracted from a previous image frame in the plurality of image frames; and
determining a pose of the image capturing device based on the first position information and the second position information of each of the matching feature points in the current image frame.

9. The electronic device according to claim 8 , wherein the extracting a plurality of matching feature points in the plurality of image frames and determining first position information of each of the matching feature points in each of the plurality of image frames comprises:
acquiring two-dimensional position information of the first position information of each of the matching feature points in each image frame; and
transforming the two-dimensional position information of

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Qiao Liang
• Lin Xiangkai
• Bao Linchao
• Ling Yonggen
• Zhu Fengming

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/248 Primary

  involving reference images or patches · CPC title

• G06T7/55

  from multiple images · CPC title