System and method for concurrent odometry and mapping

Patent metadata

Field	Value
Publication number	US-10802147-B2
Application number	US-201715595617-A
Country	US
Kind code	B2
Filing date	May 15, 2017
Priority date	May 18, 2016
Publication date	Oct 13, 2020
Grant date	Oct 13, 2020

Abstract

An electronic device tracks its motion in an environment while building a three-dimensional visual representation of the environment that is used to correct drift in the tracked motion. A motion tracking module estimates poses of the electronic device based on feature descriptors corresponding to the visual appearance of spatial features of objects in the environment. A mapping module builds a three-dimensional visual representation of the environment based on a stored plurality of maps, and feature descriptors and estimated device poses received from the motion tracking module. The mapping module provides the three-dimensional visual representation of the environment to a localization module, which identifies correspondences between stored and observed feature descriptors. The localization module performs a loop closure by minimizing the discrepancies between matching feature descriptors to compute a localized pose. The localized pose corrects drift in the estimated pose generated by the motion tracking module.

First claim

What is claimed is:

1. A method comprising:
estimating, at a front-end motion tracking module of an electronic device, a current pose of the electronic device based on feature descriptors of one or more spatial features representing an object in an environment of the electronic device, wherein the feature descriptors are generated based on images captured from one or more visual sensors and non-visual data from one or more non-image sensors, and wherein estimating the current pose comprises:
generating linearization points based on the generated feature descriptors; and
solving a non-linear estimation of the one or more spatial features based on the generated linearization points and previously-generated linearization points based on feature descriptors from a stored history of estimated poses, wherein solving the non-linear estimation comprises holding the previously-generated linearization points to a set of fixed values;

generating, at a back-end mapping module of the electronic device, a three-dimensional representation of the environment of the electronic device based on a plurality of stored maps that were previously generated by the electronic device during prior mapping sessions and which comprise stored feature descriptors, the stored maps forming a limited history of the environment and tracked motion of the electronic device;
localizing, at a localization module of the electronic device, the estimated current pose based on the three-dimensional representation of the environment to generate a localized pose; and
updating the estimated current pose at a first, relatively higher rate, based on non-visual data from the one or more non-image sensors and updating the localized pose at a second, relatively lower rate, based on the generated feature descriptors.

2. The method of claim 1 , further comprising updating the three-dimensional representation by adding the generated feature descriptors to the three-dimensional representation of the environment.

3. The method of claim 2 , wherein adding the generated feature descriptors comprises generating linearization points based on the generated feature descriptors and solving a non-linear estimation of the three-dimensional representation based on the linearization points and previously-generated linearization points based on the stored feature descriptors, wherein previously-generated linearization points are considered variable.

4. The method of claim 2 , wherein the generated feature descriptors are added to the three-dimensional representation of the environment periodically.

5. The method of claim 2 , further comprising building a bundle adjustment problem comprising the feature descriptors of the updated three-dimensional representation and solving the bundle adjustment problem.

6. The method of claim 1 , wherein localizing the estimated current pose comprises matching the generated feature descriptors to the stored feature descriptors.

7. The method of claim 6 , wherein localizing the estimated current pose further comprises identifying discrepancies between the matched generated feature descriptors and stored feature descriptors and minimizing the discrepancies.

Assignees

• Google Llc

Inventors

• Nerurkar Esha
• Lynen Simon
• Zhao Sheng

Classifications

• G06T7/55 Primary

  from multiple images · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T17/05

  Geographic models · CPC title

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title