Creating a topological map for localization in augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-10909760-B2
Application number	US-201514705983-A
Country	US
Kind code	B2
Filing date	May 7, 2015
Priority date	Apr 18, 2014
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

An augmented reality system comprises one or more sensors configured for capturing a set of data indicative of an emotional state of a user of the augmented reality system, and a processor configured for discerning the emotional state of the user based at least in part on the captured set of data, and rendering the discerned emotional state on an avatar of the user. The avatar can be displayed as a virtual object when viewed through one or more augmented reality display systems.

First claim

What is claimed is:

1. An augmented reality (AR) system, comprising:
a cloud database storing a passable world model that is accessible by respective AR display systems of different users of the AR system, the passable world model comprising a plurality of color histogram fingerprint data sets for respective known locations within the passable world model, wherein respective geometric information for respective known locations are associated with respective color histogram fingerprint data sets in the passable world model and uniquely characterize respective physical spaces at respective known locations; and
a processor configured for localizing a particular user of an AR display system within the passable world model by:
receiving data comprising an image of a physical space acquired by a camera of an AR display system of a particular user of the AR system at a particular location, wherein respective AR display systems of respective users are in communication with or include at least one camera to capture respective images of respective physical spaces at respective locations,
generating a color histogram based on the acquired image, wherein generating the color histogram comprises modifying a format of the acquired image;
accessing the cloud database through a network connection,
performing a comparison of the generated color histogram and color histogram fingerprint data sets in the cloud database,
identifying the particular known location of the user based on the comparison, and
retrieving a set of the geometric information from the cloud database pertaining to a plurality of real objects at the particular location, wherein the color histogram fingerprint data set that uniquely identifies the identified particular location has less data than the retrieved set of geometric information,
the AR display system of the particular user being configured for displaying to the particular user virtual content comprising the retrieved geometric information in relation to a plurality of real objects at the particular location.

2. The AR system of claim 1 , wherein each of the color histogram fingerprint data sets fingerprint data sets is a combination of different types of data.

3. The AR system of claim 1 , wherein the particular location of the particular user is a room within a building.

4. The AR system of claim 1 , wherein the processor is configured for constructing a topological map based at least in part on the color histogram fingerprint data sets.

5. The AR system of claim 4 , wherein each of the color histogram fingerprint data sets represents a node of the constructed topological map.

6. The AR system of claim 5 , wherein the processor is configured for connecting a first node to a second node of the constructed topological map based on the first node and the second node havinq at least one shared AR device in common.

7. The AR system of claim 4 , wherein the topological map is layered over geometric information of the real world.

8. The AR system of claim 1 , wherein the captured data comprises an image of the user's surroundings.

9. The AR system of claim 1 , wherein the one or more sensors comprises an image-based sensor.

10. The AR system of claim 1 , the processor being configured for receiving a set of additional data based at least in part upon the identified particular location, wherein the set of additional data corresponds to the matching color histogram fingerprint data set.

11. The AR system of claim 1 , wherein the processor is configured for adding the captured data to the cloud database as a color histogram fingerprint data set based on the captured data not matching any of the color histogram fingerprint data sets stored in the cloud database.

12. The AR system of claim 1 , wherein the particular location is the location at which the data was captured.

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A
• Hand Randall E
• Chan Richmond B

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/80

  Recognising image objects characterised by unique random patterns · CPC title