Systems and methods for GPS-based and sensor-based relocalization

Patent metadata

Field	Value
Publication number	US-11113894-B1
Application number	US-202017018753-A
Country	US
Kind code	B1
Filing date	Sep 11, 2020
Priority date	Sep 11, 2020
Publication date	Sep 7, 2021
Grant date	Sep 7, 2021

Abstract

Systems are configured for performing GPS-based and sensor-based relocalization. During the relocalization, the systems are configured to obtain radio-based positioning data indicating an estimated position of the system within a mapped environment. The systems are also configured to identify, based on the estimated position, a subset of keyframes of a map of the mapped environment, wherein the map of the mapped environment includes a plurality of keyframes captured from a plurality of locations within the mapped environment, and the plurality of keyframes are associated with anchor points identified within the mapped environment. The systems are further configured to perform relocalization within the mapped environment based on the subset of keyframes.

First claim

We claim:

1. A system for facilitating relocalization within a mapped environment, the system comprising:
one or more processors; and
one or more hardware storage devices storing computer-executable instructions that are executable by the one or more processors to configure the system to facilitate relocalization within a mapped environment and by at least configuring the system to perform the following:
obtain radio-based positioning data indicating an estimated position of the system within the mapped environment;
identify, based on the estimated position, a subset of keyframes of a map of the mapped environment, the map of the mapped environment comprising a plurality of keyframes captured from a plurality of locations within the mapped environment, the plurality of keyframes being associated with anchor points identified within the mapped environment; and
relocalize the system within the mapped environment based on the subset of keyframes.

2. The system of claim 1 , wherein the radio-based positioning data comprises global positioning system (GPS) data obtained by a GPS device associated with the system.

3. The system of claim 2 , wherein the system comprises a head-mounted display (HMD).

4. The system of claim 2 , wherein:
the system comprises a user instrument associated with an HMD,
the GPS device is mounted on the HMD, and
the user instrument omits a separate GPS device mounted thereon.

5. The system of claim 1 , wherein identifying the subset of keyframes of the map of the mapped environment comprises:
defining a search space within the map based on the estimated position; and
identifying one or more keyframes of the plurality of keyframes, the one or more keyframes being captured from one or more positions within the mapped environment that correspond to the search space.

6. The system of claim 5 , wherein the defining of the search space is based on barometer data associated with the system for defining an altitude of the search space.

7. The system of claim 5 , wherein the defining of the search space is based on a confidence measure associated with the radio-based positioning data.

8. The system of claim 5 , wherein the defining of the search space is based on a geometry of a region of the mapped environment proximate to the estimated position.

9. The system of claim 5 , wherein the computer-executable instructions are executable by the one or more processors to configure the system to:
obtain an estimated orientation of the system based on inertial tracking data from one or more inertial tracking components associated with the system.

10. The system of claim 9 , wherein the one or more inertial tracking components comprise an accelerometer, a magnetometer, or a gyroscope.

11. The system of claim 9 , wherein identifying the subset of keyframes of the map of the mapped environment comprises:
identifying one or more keyframes within the search space that meet or exceed a threshold orientation similarity to the estimated orientation.

12. The system of claim 11 , wherein the threshold orientation similarity is based on a confidence measure associated with the inertial tracking data.

13. The system of claim 1 , wherein relocalizing the system within the mapped environment based on the subset of keyframes comprises:
determining a keyframe of highest correspondence to one or more visual tracking images captured by one or more cameras associated with the system, wherein determining the keyframe of highest correspondence comprises determining feature correspondences between the one or more visual tracking images and anchor points associated with each keyframe of the subset of keyframes; and
estimating a pose of the system within the mapped environment relative to the keyframe of highest correspondence.

14. The system of claim 1 , wherein the computer-executable instructions are executable by the one or more processors to configure the system to refrain from relocalizing the system within the mapped environment based on keyframes of the plurality of keyframes of the map that are not included in the subset of keyframes.

15. The system of claim 1 , wherein the computer-executable instructions are executable by the one or more processors to configure the system to, in response to failing to relocalize the system within the mapped environment based on the subset of keyframes, modify the subset of keyframes to include additional keyframes of the plurality of keyframes.

16. A system for determining a pose of a system relative to a mapped environment, the system comprising:
one or more processors; and
one or more hardware storage devices storing computer-executable instructions that are executable by the one or more processors to configure the system to determine a pose of a system relative to a mapped environment and by at least configuring the system to perform the following:
generate head tracking data associated with the system, the head tracking data being generated based on at least (1) visual tracking data obtained using one or more cameras associated with the system and (2) inertial tracking data obtained using one or more inertial tracking components associated with the system;
obtain radio-based positioning data generated using one or more radio-based positioning components associated with the system;
limit a search space within a map of the mapped environment based on the radio-based positioning data; and
determine a pose of the system within the search space using at least a portion of the head tracking data as input.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Price Raymond Kirk
• Bleyer Michael
• Edmonds Christopher Douglas

Classifications

• G01C21/206 Primary

  specially adapted for indoor navigation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G01S19/485

  whereby the further system is an optical system or imaging system · CPC title