Reducing stresses in the passable world model in augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-10665018-B2
Application number	US-201514705986-A
Country	US
Kind code	B2
Filing date	May 7, 2015
Priority date	Apr 18, 2014
Publication date	May 26, 2020
Grant date	May 26, 2020

Abstract

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

First claim

What is claimed is:

1. A method of displaying augmented reality, comprising:
capturing one or more sets of map points from the real world through a plurality of augmented reality systems, wherein the one or more sets of map points include a map point that corresponds to a real world object;
determining parameterized geometry of the real world object at least by deriving, via a least two-dimensional and three-dimensional data fusion on at least a set of sparse points, a structure of and one or more properties about a parametric primitive that represents the real world object with an adjustable size and at least by parameterizing geometry of the real world object into the parameterized geometry that includes parametric geometry, wherein the parametric geometry includes geometric definitions and polygonal definitions of the real world object;
constructing a geometric map of the real world based at least in part on the one or more sets of map points and the parameterized geometry of the real world object;
estimating, at one or more object recognizers of a second augmented reality system, a future position, a dynamic movement, or a dynamic orientation of the real world object with at least the parameterized geometry and semantic information attached to the parametric primitive, rather than via repeatedly captures of movements and orientations of the real world object; and
displaying, at the second augmented reality system, a virtual object that represents the real world object at the future position, the dynamic movement, or the dynamic orientation estimated by the one or more object recognizers.

2. The method of claim 1 , further comprising identifying a point of stress in the geometric map, wherein a strength of a connection between two nodes of the geometric map corresponds to a number of shared map points between the two nodes.

3. The method of claim 2 , wherein the point of stress is identified based at least in part on information retrieved from a topological map.

4. The method of claim 3 , further comprising:
determining loop closure stress at one or more map points using at least the topological map.

5. The method of claim 4 , further comprising:
layering the topological map on the geometric map by layering topological map nodes of the topological map on top of geometric map nodes in the geometric map;
determining whether two geometric map nodes in the geometric map are to be a same geometric map node at least by examining one or more corresponding topological map nodes; and
identifying the loop closure stress in the geometric map when a single topological map node corresponds to the two geometric map nodes.

6. The method of claim 4 , further comprising:
identifying a first physical space in real world;
creating a first fingerprint or signature of a first physical space in real world at least by processing first data captured by a first augmented reality system of the plurality of augmented reality systems, rather than using positioning system location data or pose tagged images;
creating the topological map at least by representing the first physical space in the real world as a first point node in the topological map and associating the first fingerprint or signature with the first point node; and
updating the topological map at least by adding a second point node to the topological map and connecting the first point node and the second point node with an edge in the topological map, wherein the edge represents connectivity between the first point node and the second point node.

7. The method of claim 6 , further comprising:
determining whether a second augmented reality system is in the first physical space at least by processing second data captured by the second augmented reality system into a second fingerprint or signature and by comparing the second fingerprint or signature to the first fingerprint or signature associated with the first point node in the topological map; and
when the second augmented reality system is determined to be in the first physical space, retrieving one or more keyframes associated with the first point node and transmitting the one or more keyframes to the second augmented reality system via a network.

8. The method of claim 2 , wherein the point of stress is identified based at least in part on a discrepancy in a location of a particular keyframe in relation to the geometric map.

9. The method of claim 2 , wherein the point of stress is identified based at least in part on a maximum residual error of the geometric map.

10. The method of claim 2 , wherein the point of stress is distributed through a bundle adjust process.

11. The method of claim 10 , further comprising:
distributing a stress at the point of stress to an area between two degrees of separation and three degrees of separation in the geometric map with respect to the point of stress.

12. The method of claim 2 , wherein a stress associated with the point of stress is radially distributed to a first wave of nodes outside a closest node that is closest to the point of stress.

13. The method of claim 12 , wherein the first wave of nodes outside of the closest node comprises a network or nodes that has a single degree of separation from the closest node that is closest to the point of stress.

14. The method of claim 12 , wherein the stress associated with point of stress is further radially distributed to second wave of nodes outside the first wave of nodes.

15. The method of claim 12 , wherein the first wave of nodes are marked when the stress is radially distributed to the first wave of nodes.

16. The method of claim 1 , further comprising:
identifying a plurality of keyframes including the keyframe, wherein the semantic information comprises one or more taxonomical descriptors for the real world object;
representing the plurality of keyframes as a plurality of nodes that includes the node; and
determining whether two nodes in the plurality of nodes are connected based in part or in whole upon whether there is at least one shared augmented reality device in common.

17. The method of claim 16 , further comprising:
determining an edge between the two nodes based in part or in whole upon a total number of shared map points between the two nodes that are determined to be connected; and
constructing the geometric map at least by inserting the plurality of keyframes as the plurality of nodes in the geometric map and connecting the plurality of nodes with the at least the edge between the two nodes.

18. The method of claim 17 ,
capturing a plurality of sets of two-dimensional map points from the real world through the plurality of augmented reality systems; and
determining a set of three-dimensional (3D) map points based in part or in whole upon the plurality of sets of two-dimensional map points, wherein
a first keyframe corresponding to a first node in the geometric map represents a first room in a real world, and a second keyframe corresponding to a second node in the geometric map represents a second room in the real world.

19. The method of claim 17 , further comprising:
representing a first edge connecting a first pair of nodes in the geometric map with a first graphical emphasis; and
representing a second edge connecting a second pair of nodes in the geometric map with a second graphical emphasis.

20. The method of claim 17 , further comprising:
determining a strength of the edge based in part or in whole upon geometric proximity between two real world locations represented by the two

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A

Classifications

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T15/10

  Geometric effects · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title

• G02B27/0101

  characterised by optical features (G02B27/0172 takes precedence) · CPC title