Multi-Sync Ensemble Model for Device Localization

Patent metadata

Field	Value
Publication number	US-2020008024-A1
Application number	US-201916455630-A
Country	US
Kind code	A1
Filing date	Jun 27, 2019
Priority date	Jun 27, 2018
Publication date	Jan 2, 2020
Grant date	—

Abstract

A system and method determine the location of a device. The device collects sensor data using one or more sensors. Based on the sensor data, one or more localization models are selected from a plurality of localization models. The selected models are applied to generate one or more potential locations. The current location of the device is determined based on the one or more potential locations.

First claim

1 . A method for determining a location of a device, the method comprising:
collecting sensor data using one or more sensors; selecting, based on the sensor data, one or more localization models from a plurality of localization models; applying the selected one or more localization models to generate one or more potential locations of the device; and determining the location of the device based on the one or more potential locations.

2 . The method of claim 1 , wherein the sensor data includes an image captured by a camera.

3 . The method of claim 1 , wherein the plurality of localization models includes at least one of: a point cloud based model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, an object recognition model, a semantic matching model, a cube matching model, a cylinder matching model, a horizon matching model, a light source matching model, and a landscape recognition model.

4 . The method of claim 1 , wherein applying the selected one or more localization models comprises applying a plurality of localization models to generate a plurality of potential locations, wherein each applied localization model generates one of the plurality of potential locations.

5 . The method of claim 4 , wherein determining the location of the device comprises calculating an average location of the plurality of potential locations.

6 . The method of claim 5 , wherein the average is a weighted average.

7 . The method of claim 1 , wherein selecting one or more localization models comprises:
determining whether the device is currently indoors or outdoors; and selecting a subset of the localization models based on whether the device is currently indoors or outdoors.

8 . The method of claim 1 , wherein selecting one or more localization models comprises:
assigning a score to each localization model, the score for a localization model indicating a likelihood of that localization model generates an accurate location; and selecting the one or more localization models based on the score.

9 . The method of claim 8 , wherein the scores are based on historic performance of the models for similar environments.

10 . The method of claim 8 , wherein the scores are based on the sensor data.

11 . A device comprising:
one or more processors; and one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the device to perform operations including: collecting sensor data using one or more sensors; selecting, based on the sensor data, one or more localization models from a plurality of localization models; applying the selected one or more localization models to generate one or more potential locations of the device; and determining a location of the device based on the one or more potential locations.

12 . The device of claim 11 , wherein the sensor data includes an image captured by a camera.

13 . The device of claim 11 , wherein the plurality of localization models includes at least one of: a point cloud based model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, an object recognition model, a semantic matching model, a cube matching model, a cylinder matching model, a horizon matching model, a light source matching model, and a landscape recognition model.

14 . The device of claim 11 , wherein applying the selected one or more localization models comprises applying a plurality of localization models to generate a plurality of potential locations, wherein each applied localization model generates one of the plurality of potential locations.

15 . The device of claim 14 , wherein determining the location of the device comprises calculating an average location of the plurality of potential locations.

16 . The device of claim 15 , wherein the average is a weighted average.

17 . The device of claim 11 , wherein selecting one or more localization models comprises:
determining whether the device is currently indoors or outdoors; and selecting a subset of the localization models based on whether the device is currently indoors or outdoors.

18 . The device of claim 11 , wherein selecting one or more localization models comprises:
assigning a score to each localization model, the score for a localization model indicating a likelihood of that localization model generates an accurate location; and selecting the one or more localization models based on the score.

19 . The device of claim 18 , wherein the scores are based on historic performance of the models for similar environments.

20 . The device of claim 18 , wherein the scores are based on the sensor data.

Assignees

• Niantic Inc

Inventors

• Hu Si Ying Diana
• Ashok Anubhav
• Turner Peter

Classifications

• H04W4/38 Primary

  for collecting sensor information · CPC title

• H04W4/025

  using location based information parameters · CPC title

• H04W4/029

  Location-based management or tracking services · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/73

  using feature-based methods · CPC title