Non-matching feature-based visual motion estimation for pose determination

Patent metadata

Field	Value
Publication number	US-2016307334-A1
Application number	US-201615131404-A
Country	US
Kind code	A1
Filing date	Apr 18, 2016
Priority date	Apr 20, 2015
Publication date	Oct 20, 2016
Grant date	—

Abstract

A method for determining a pose of a camera includes obtaining both a first image of a scene and a second image of the scene, where both the first and second images are captured by the camera. A first set of features is extracted from the first image and a second set of features is extracted from the second set of features. The method includes calculating a value of a visual-motion parameter based on the first set of features and the second set of features without matching features of the first set with features of the second set. The method also includes determining the pose of the camera based, at least, on the value of the visual motion parameter.

First claim

What is claimed is:

1 . A method for determining a pose of a camera, the method comprising:
obtaining a first image of a scene, wherein the first image is captured by the camera; obtaining a second image of a scene, wherein the second image is captured by the camera; extracting a first set of features from the first image; extracting a second set of features from the second image; calculating a value for a visual-motion parameter based on the first set of features and the second set of features without matching features of the first set with features of the second set; and determining the pose of the camera based, at least, on the value of the visual-motion parameter.

2 . The method of claim 1 , wherein calculating the value of the visual-motion parameter based on the first set of features and the second set of features without matching features of the first set with features of the second set comprises:
deriving a cost function for matching each feature in the first set of features with each feature in the second set of features; creating a matrix encoding the cost of matching features from the first set to the second set for a plurality of possible values of the visual-motion parameter; and optimizing the cost function to determine the value of the visual-motion parameter.

3 . The method of claim 2 , wherein the matrix is arranged into rows, corresponding the first set of features, and into columns, corresponding to the second set of features.

4 . The method of claim 2 , wherein the matrix further encodes additional costs corresponding to a possibility that at least one feature of the first set of features or at least one feature of the second set of features does not have a match.

5 . The method of claim 2 , wherein deriving the cost function comprises determining a permanent of the matrix.

6 . The method of claim 5 , wherein an approximation method is used to estimate the permanent of the matrix.

7 . The method of claim 5 , wherein optimizing the cost function comprises:
selecting a plurality of candidate motion models; calculating a plurality of cost function values of the cost function for the plurality of candidate motion models; and determining the value of the visual-motion parameter based on the plurality of cost function values.

8 . The method of claim 7 , wherein selecting the plurality of candidate motion models comprises:
applying a linear approximation for the permanent of the matrix; obtaining a first approximation for the value of the visual-motion parameter by solving a version of the cost function; and adding the first approximation to the plurality of candidate motion models.

9 . The method of claim 8 , wherein determining the value of the visual-motion parameter based on the plurality of cost function values comprises selecting a value of the visual-motion parameter corresponding to a best cost function value of the plurality of cost function values.

10 . The method of claim 1 , wherein the visual-motion parameter is at least one parameter selected from the group consisting of a homography, a fundamental matrix, a rigid transformation between two sets of points, essential matrices, affine transformation, and projective transformation.

11 . A mobile device, the mobile device comprising:
a camera; memory adapted to store program code; and a processing unit coupled to the memory to access and execute instructions included in the program code to direct the mobile device to:
capture a first image of a scene with the camera;
capture a second image of the scene with the camera;
extract a first set of features from the first image;
extract a second set of features from the second image;
calculate a value for a visual-motion parameter based on the first set of features and the second set of features without matching features of the first set with features of the second set; and
determine a pose of the camera based, at least, on the value of the visual-motion parameter.

12 . The mobile device of claim 11 , wherein the instructions to calculate the value of the visual-motion parameter based on the first set of features and the second set of features without matching features of the first set with features of the second set comprises instructions to direct the mobile device to:
derive a cost function for matching each feature in the first set of features with each feature in the second set of features; create a matrix encoding the cost of matching features from the first set to the second set for a plurality of possible values of the visual-motion parameter; and optimize the cost function to determine the value of the visual-motion parameter.

13 . The mobile device of claim 12 , wherein the matrix is arranged into rows, corresponding the first set of features, and into columns, corresponding to the second set of features.

14 . The mobile device of claim 12 , wherein the matrix further encodes additional costs corresponding to a possibility that at least one feature of the first set of features or at least one feature of the second set of features does not have a match.

15 . The mobile device of claim 12 , wherein the instructions to derive the cost function comprises at least one instruction to direct the mobile device to determine a permanent of the matrix.

16 . The mobile device of claim 15 , wherein an approximation method is used to estimate the permanent of the matrix.

17 . The mobile device of claim 15 , wherein the instructions to optimize the cost function comprises instructions to direct the mobile device to:
select a plurality of candidate motion models; calculate a plurality of cost function values of the cost function for the plurality of candidate motion models; and determine the value of the visual-motion parameter based on the plurality of cost function values.

18 . The mobile device of claim 17 , wherein the instructions to select the plurality of candidate motion models comprises instructions to direct the mobile device to:
apply a linear approximation for the permanent of the matrix; obtain a first approximation for the value of the visual-motion parameter by solving a version of the cost function; and add the first approximation to the plurality of candidate motion models.

19 . The mobile device of claim 18 , wherein the instructions to determine the value of the visual-motion parameter based on the plurality of cost function values comprises instructions to direct the mobile device to select a value of the visual-motion parameter corresponding to a best cost function value of the plurality of cost function values.

20 . The mobile device of claim 11 , wherein the visual-motion parameter is at least one parameter selected from the group consisting of a homography, a fundamental matrix, a rigid transformation between two sets of points, essential matrices, affine transformation, and projective transformation.

21 . A non-transitory computer-readable medium including program code stored thereon for determining a pose of a camera, the program code comprising instructions to:
obtain a first image of a scene, wherein the first image is captured by the camera; obtaining a second image of a scene, wherein the second image is captured by the camera; extract a first set of features from the first image; extract a second set of features from the second image; calculate a value for a visual-motion parameter based on the first set of features and the second set of features without matchi

Assignees

• Qualcomm Inc

Inventors

• Dos Santos Mendonca Paulo Ricardo

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T7/0022

  Physics · mapped topic

• G06K9/20

  Physics · mapped topic

• G06T7/2033 Primary

  Physics · mapped topic

• G06T2207/30244

  Camera pose · CPC title