Crowd-sourced creation and updating of area description file for mobile device localization

Patent metadata

Field	Value
Publication number	US-9811734-B2
Application number	US-201514708877-A
Country	US
Kind code	B2
Filing date	May 11, 2015
Priority date	May 11, 2015
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

A computing system includes a network interface, a first datastore, a second datastore, and a merge module. The merge module is to receive a set of one or more area description files from a set of one or more first mobile devices. Each area description file represents a point cloud of spatial features detected by a corresponding first mobile device at an area. The computing system further includes a localization module and a query module. The localization generation module is to generate a localization area description file for the area from the set of one or more area description files and to store the localization area description file in the second datastore. The localization area description file represents a point cloud of spatial features for the area. The query module is to provide the localization area description file to a second mobile device via the network interface.

First claim

What is claimed is:

1. A method comprising:
receiving, at a computing system, a set of one or more area description files from a set of one or more first mobile devices, each area description file representing a point cloud of spatial features detected by a corresponding first mobile device at an area;
generating, at the computing system, a localization area description file for the area from the set of one or more area description files, the localization area description file representing a point cloud of spatial features for the area; and
providing the localization area description file from the computing system to a second mobile device, wherein:
the set of one or more area description files comprises a plurality of area description files; and
generating the localization area description file comprises:
identifying the plurality of area description files as being associated with the area;
deduplicating the point clouds represented by the plurality of area description files;
determining a relative alignment of the point clouds represented by the plurality of area description files; and
incorporating the point clouds in the localization area description file.

2. The method of claim 1 , further comprising:
performing a localization process at the second mobile device using the localization area description file.

3. A method comprising:
receiving, at a computing system, a set of one or more area description files from a set of one or more first mobile devices, each area description file representing a point cloud of spatial features detected by a corresponding first mobile device at an area;
generating, at the computing system, a localization area description file for the area from the set of one or more area description files, the localization area description file representing a point cloud of spatial feature for the area; and
providing the localization area description file from the computing system to a second mobile device,
wherein each area description file of the set of one or more area description files comprises:
spatial feature data specifying, for each spatial feature of the point cloud represented by the area description file, a multidimensional coordinate of the spatial feature based on a corresponding coordinate frame;
statistical data representing, for each spatial feature, a visual appearance of the spatial feature independent of angle of view or lighting conditions; and
sensor data captured from one or more non-image sensors concurrent with detection of the spatial features of the point cloud.

4. The method of claim 3 , wherein generating the localization area description file comprises:
incorporating at least a subset of the spatial feature data and the statistical data of each area description file of the set in the localization area description file; and
excluding the sensor data of each area description file in the set from the localization area description file.

5. The method of claim 1 , wherein generating the localization area description file further comprises:
eliminating spatial features from the point clouds based on at least one stability criterion.

6. The method of claim 1 , wherein generating the localization area description file further comprises:
identifying a plurality of sub-areas based on the point clouds; and
generating a local area description file for each sub-area from the spatial features of the point clouds that are located in the sub-area.

7. The method of claim 1 , further comprising:
receiving, at the computing system, feedback data from the second mobile device, the feedback data identifying whether one or more of the spatial features represented by the localization area description file was detected by the second mobile device while in the area;
updating, at the computing system, the localization area description file based on the feedback data to generate an updated localization area description file; and
providing the updated localization area description file from the computing system to a third mobile device.

8. A method comprising;
receiving, at a computing system, a set of one or more area description files from a set of one or more first mobile devices, each area description file representing a point cloud of spatial feature detected by a corresponding first mobile device at an area;
generating, at the computing system, a localization area description file for the area from the set of one or more description files, the localization area description file representing a point cloud of spatial feature for the area;
providing the localization area description file from the computing system to a second mobile device;
receiving, at the computing system, feedback data from the second mobile device, the feedback data identifying whether one or more of the spatial feature represented by the localization area description file was detected by the second mobile device while in the area;
updating, at the computing system, the localization area description file based on the feedback data to generate an update localization area description file; and
providing the update localization area description file from the computing system to a third mobile device,
wherein updating the localization area description file comprises:
maintaining a score for each spatial feature represented by the local area description file; and
for a spatial feature represented by the localization area description file:
increasing a score of the spatial feature in response to the feedback data indicating that the spatial feature was detected by the second mobile device;
decreasing the score of the spatial feature in response to the feedback data indicating that the spatial feature was not detected by the second mobile device; and
selectively removing the spatial feature from the localization area description file based on a comparison of the score to at least one threshold.

9. A method comprising;
receiving, at a computing system, a set of one or more area description files from a set of one or more first mobile devices, each area description file representing a point cloud of spatial feature detected by a corresponding first mobile device at an area;
generating, at the computing system, a localization area description file for the area from the set of one or more area description files, the localization area description file representing a point cloud of spatial feature for the area;
providing the localization area description file from the computing system to a second mobile device;
receiving, at the computing system, feedback data from the second mobile device, the feedback data identifying whether one or more of the spatial feature represented by the localization area description file was detected by the second mobile device while in the area;
updating, at the computing system, the localization area description file based on the feedback data to generate an update localization area description file; and
providing the update localization area description file from the computing system to a third mobile device, wherein:
the feedback data identifies a spatial feature detected by the second mobile device and which was not represented by the localization area description file; and
wherein updating the localization area description file comprises:
increasing a score associated with the spatial feature detected by the second mobile device; and
selectively modifying the localization area description file to include the spatial feature detected by the second mobile device responsive to a comparison of the score and at least one threshold.

10. A computing system comprising;
a network interface to couple to a plurality of mobile devices;
a f

Assignees

• Google Inc

Inventors

• Williams Brian Patrick
• Hickman Ryan Michael
• Tu Laurent
• Nerurkar Esha
• Lynen Simon

Classifications

• G01C21/206 Primary

  specially adapted for indoor navigation · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06F16/5838

  using colour · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title