Using a map of the world for augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-10115232-B2
Application number	US-201514705856-A
Country	US
Kind code	B2
Filing date	May 6, 2015
Priority date	Apr 18, 2014
Publication date	Oct 30, 2018
Grant date	Oct 30, 2018

Abstract

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

First claim

What is claimed is:

1. An augmented reality system, comprising:
a first individual augmented reality display system corresponding to a first location, wherein the first individual augmented reality display system is configured for capturing a first set of geometric map points pertaining to the first location;
a second individual augmented reality display system corresponding to a second location, wherein the second individual augmented reality display system is configured for capturing a second set of geometric map points pertaining to the second location;
a third individual augmented reality display system configured for capturing data pertaining to a particular location of a user; and
a server comprising a processor configured for receiving the first set of geometric map points from the first individual augmented reality display system and the second set of geometric map points from the second individual augmented reality display system, and constructing at least a portion of a passable geometric map of the real world comprising the first and second locations using a topological graph to spatially stitch the first and second sets of geometric map points together into a single larger coherent passable geometric map, wherein the processor is further configured for storing the single larger coherent passable geometric map in a database, for localizing the user within the single larger coherent map of the real world by comparing the captured data to the topological map, and retrieving a third set of the geometric points from the database pertaining to a plurality of real objects at the particular location of the user;
wherein the third individual augmented reality display system is configured for displaying virtual content to the user in relation to the plurality of real objects.

2. The augmented reality system of claim 1 , wherein the first augmented reality display system is a head-mounted augmented reality display system.

3. The augmented reality display system of claim 1 , wherein the first augmented reality display system is a room-based sensor system.

4. The augmented reality display system of claim 1 , wherein the server is configured for transmitting the constructed at least portion of the passable geometric map of the real world to at least one of the first and second augmented reality display systems.

5. The augmented reality display system of claim 4 , wherein at least one of the first and second augmented reality display systems is configured for projecting a virtual object based at least in part on the constructed at least portion of the passable geometric map of the real world.

6. The augmented reality display system of claim 1 , wherein the captured first set of geometric data points and the second set of geometric data points are at least an images captured at the first and second location.

7. The augmented reality display system of claim 1 , wherein the captured first set of geometric data points and the second set of geometric data points correspond to sensor data.

8. The augmented reality display system of claim 1 , wherein the processor is configured for extracting a set of map points from the first set of geometric data points and the second set of geometric data points captured from the first and second augmented reality display systems, and for using the set of map points to construct the at least portion of the passable geometric map of the real world.

9. The augmented reality display system of claim 1 , wherein the server is configured for transmitting a part of the at least portion of the passable geometric map of the real world corresponding to the first augmented reality display system to the second augmented reality display system.

10. The augmented reality display system of claim 1 , wherein the captured first set of geometric data points and the second set of geometric data points comprise pose tagged images corresponding to the first location.

11. The augmented reality display system of claim 1 , wherein the captured first set of geometric data points or the second set of geometric data points comprise pose information of the first and second augmented reality display systems, wherein the at least portion of the passable geometric map of the real world is constructed based at least in part on the pose information.

12. The augmented reality display system of claim 1 , wherein the topological graph comprises color histograms, and wherein the processor is configured for using the color histograms to stitch the received first and second sets of geometric map points together into the single larger coherent passable geometric map.

13. The augmented reality display system of claim 1 , wherein the topological graph comprises a plurality of nodes and a plurality of connections between the nodes.

14. The augmented reality display system of claim 13 , wherein each of the nodes represents a particular location in the real world.

15. The augmented reality display system of claim 1 , wherein the topological graph is layered over the geometric map data.

16. The augmented reality display system of claim 13 , wherein the topological graph comprises information characterizing strengths of the connections between the nodes, wherein the received captured data is stitched together based on the received strengths of the connections between the nodes.

17. The augmented reality display system of claim 16 , wherein the strengths of the connections are based on the number of shared captured data between the first individual augmented reality display system and the second individual augmented reality display system.

18. The augmented reality display system of claim 16 , wherein the strengths of the connections are based on the geographical proximity between the nodes.

19. The augmented reality display system of claim 1 , wherein the processor is configured for using the topological graph to identify where to stitch the received captured data together.

20. The augmented reality display system of claim 1 , the database configured for storing a plurality of fingerprint data sets and the captured first set of geometric data points and the second set of geometric data points, the plurality of fingerprint data sets respectively characterizing physical spaces uniquely identifying the first and second locations;
wherein the processor is configured for constructing the topological map based at least in part on the fingerprint data sets, comparing the captured data to the topological map by comparing the captured data to the plurality of fingerprint data sets, and identifying the particular location as one of the first and second locations, wherein the fingerprint data set that uniquely identifies the identified location has less data than the retrieved third set of the geometric points.

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A
• Hand Randall E

Classifications

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/304

  For headphones · CPC title

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• A63F13/577

  using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars · CPC title