Device pose estimation using 3d line clouds

Patent metadata

Field	Value
Publication number	US-2020005486-A1
Application number	US-201816025120-A
Country	US
Kind code	A1
Filing date	Jul 2, 2018
Priority date	Jul 2, 2018
Publication date	Jan 2, 2020
Grant date	—

Abstract

Computing devices and methods for estimating a pose of a user computing device are provided. In one example a 3D map comprising a plurality of 3D points representing a physical environment is obtained. Each 3D point is transformed into a 3D line that passes through the point to generate a 3D line cloud. A query image of the environment captured by a user computing device is received, the query image comprising query features that correspond to the environment. Using the 3D line cloud and the query features, a pose of the user computing device with respect to the environment is estimated.

First claim

1 . A computing device, comprising:
a processor; and a memory holding instructions executable by the processor to:
obtain a three-dimensional map comprising a plurality of three-dimensional points representing a physical environment;
transform each three-dimensional point into a three-dimensional line that passes through the point to generate a three-dimensional line cloud;
receive a query image of the physical environment captured by a user computing device, the query image comprising a plurality of query features that correspond to at least a portion of the physical environment; and
using the three-dimensional line cloud and the plurality of query features, estimate a pose of the user computing device with respect to the physical environment.

2 . The computing device of claim 1 , wherein the instructions are executable to, for each of the three-dimensional lines, randomly select a direction for the three-dimensional line.

3 . The computing device of claim 2 , wherein the instructions are executable to:
store the three-dimensional line cloud comprising the randomly selected directions; and use the three-dimensional line cloud with subsequent query features from a subsequent query image of the physical environment to estimate a subsequent pose of another computing device with respect to the physical environment.

4 . The computing device of claim 2 , wherein each of the randomly selected directions is selected from a predetermined number of directions.

5 . The computing device of claim 1 , wherein the instructions are executable to:
designate an anchor location within a three-dimensional point cloud comprising the plurality of three-dimensional points; and for each of the three-dimensional lines, select a direction for the three-dimensional line that passes through the anchor location.

6 . The computing device of claim 1 , wherein the instructions are executable to:
designate a plurality of anchor locations within a three-dimensional point cloud comprising the plurality of three-dimensional points; and for each of the three-dimensional lines, select a direction for the three-dimensional line that passes through one of the plurality of anchor locations.

7 . The computing device of claim 1 , wherein the instructions are executable to:
receive a plurality of query images of the physical environment captured by the user computing device; using the plurality of query images, generate a query three-dimensional map comprising a plurality of query three-dimensional points representing the physical environment; and using the three-dimensional line cloud and the query three-dimensional map, estimate a pose of the user computing device with respect to the physical environment.

8 . The computing device of claim 7 , wherein estimating the pose of the user computing device with respect to the physical environment using the plurality of three-dimensional points computed from the plurality of query images comprises finding an absolute pose of a generalized camera representing the three-dimensional line cloud with respect to the plurality of query three-dimensional points in the query three-dimensional map.

9 . The computing device of claim 1 , wherein the instructions are executable to, after transforming the plurality of three-dimensional points into the three-dimensional lines, discard locations of the plurality of three-dimensional points within the three-dimensional map.

10 . The computing device of claim 1 , wherein the instructions are executable to:
determine that a subset of the three-dimensional points is located on a plane; and for each three-dimensional point of the subset of three-dimensional points, select a direction for the three-dimensional line passing through the point that is coplanar with the plane.

11 . The computing device of claim 1 , wherein estimating the pose of the user computing device with respect to the physical environment using query features extracted from the query image captured by an image capture device of the user computing device comprises finding a relative pose of the image capture device with respect to a generalized camera representing the three-dimensional line cloud.

12 . A method, comprising:
obtaining a three-dimensional map comprising a plurality of three-dimensional points representing a physical environment; transforming each three-dimensional point into a three-dimensional line that passes through the point to generate a three-dimensional line cloud; receiving a query image of the physical environment captured by a user computing device, the query image comprising a plurality of query features that correspond to at least a portion of the physical environment; and using the three-dimensional line cloud and the plurality of query features, estimating a pose of the user computing device with respect to the physical environment.

13 . The method of claim 11 , further comprising, for each of the three-dimensional lines, randomly selecting a direction for the three-dimensional line.

14 . The method of claim 12 , further comprising selecting the randomly selected directions from a predetermined number of directions.

15 . The method of claim 12 , further comprising:
storing the three-dimensional line cloud comprising the randomly selected directions; and using the three-dimensional line cloud with subsequent query features from a subsequent query image of the physical environment to estimate a subsequent pose of another computing device with respect to the physical environment.

16 . The method of claim 11 , further comprising:
designating a plurality of anchor locations within a three-dimensional point cloud comprising the plurality of three-dimensional points; and for each of the three-dimensional lines, selecting a direction for the three-dimensional line that passes through one of the plurality of anchor locations.

17 . The method of claim 11 , further comprising:
receiving a plurality of query images of the physical environment captured by the user computing device; using the plurality of query images, generating a query three-dimensional map comprising a plurality of query three-dimensional points representing the physical environment; and using the three-dimensional line cloud and the query three-dimensional map, estimating a pose of the user computing device with respect to the physical environment.

18 . The method of claim 11 , further comprising, after transforming the plurality of three-dimensional points into the three-dimensional lines, discarding locations of the plurality of three-dimensional points within the three-dimensional map.

19 . The method of claim 11 , further comprising:
determining that a subset of the three-dimensional points is located on a plane; and for each three-dimensional point of the subset of three-dimensional points, selecting a direction for the three-dimensional line passing through the point that is coplanar with the plane.

20 . A computing device, comprising:
a processor; and a memory holding instructions executable by the processor to:
obtain a three-dimensional map comprising a plurality of three-dimensional points representing a physical environment;
transform each three-dimensional point into a three-dimensional plane that contains the point to generate a three-dimensional plane cloud;
receive at least one query image of the physical environment captured by an image capture device of a user computing device, the at least one query image comprising a plurality of query featur

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Sinha Sudipta Narayan
• Speciale Pablo Alejandro
• Kang Sing Bing
• Pollefeys Marc Andre Leon

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T7/337

  involving reference images or patches · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title