Stress reduction in geometric maps of passable world model in augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-9984506-B2
Application number	US-201514705989-A
Country	US
Kind code	B2
Filing date	May 7, 2015
Priority date	Apr 18, 2014
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

First claim

What is claimed is:

1. An augmented or virtual reality system, comprising:
a set of map points captured in a plurality of existing keyframes from a real world by a set of augmented or virtual reality display systems;
a data structure to receive or store the set of map points captured from the real world;
a processor communicatively coupled to the data structure to construct or update a geometric map of the real world based at least in part on the set of map points, wherein
a node in the geometric map comprises a keyframe that captures at least a subset of map points of the set of map points, and
a strength of a connection between two nodes in the geometric map corresponds to a number of map points shared between the two nodes;

the processor further configured to generate or update a simplified map corresponding to the geometric map at least by representing the real world with a plurality of point nodes corresponding to the plurality of existing keyframes and connecting the plurality of point nodes with a set of edges;
the processor further configured to determine and position a virtual keyframe in relation to a normal direction of at least one existing keyframe of the plurality of existing keyframes, and
a head worn or mounted display coupled to the processor, comprising a waveguide, and configured to project light beams to at least one eye of a user based at least in part upon the geometric map, the simplified map, and the virtual keyframe that is determined and positioned in relation to a normal direction of at least one existing keyframe of the plurality of existing keyframes.

2. The augmented or virtual reality system of claim 1 , wherein the processor is configured to determine one or more new map points at least by rendering rays or cones from the virtual keyframe to at least some map points in the set of map points based in part or in whole upon one or more points of intersections formed by the rays or cones.

3. The augmented or virtual reality system of claim 2 , wherein the processor is further configured to identify a point of stress in the geometric map, and the point of stress is identified based at least in part on information retrieved from a topological map.

4. The augmented or virtual reality system of claim 2 , wherein the point of stress is identified based at least in part on a discrepancy in a location of a particular keyframe in relation to the geometric map.

5. The augmented or virtual reality system of claim 2 , wherein the point of stress is identified based on a maximum residual error of the geometric map.

6. The augmented or virtual reality system of claim 2 , wherein a stress associated with the point of stress is distributed through a bundle adjust process.

7. The augmented or virtual reality system of claim 2 , wherein a stress associated with the identified point of stress is radially distributed to a first wave of nodes outside the node closest to the point of stress.

8. The augmented or virtual reality system of claim 7 , wherein the first wave of nodes outside of the node comprises a network or nodes that have a single degree of separation from the node closest to the point of stress.

9. The augmented or virtual reality system of claim 7 , wherein a stress associated with the point of stress is further radially distributed to second wave of nodes outside the first wave of nodes.

10. The augmented or virtual reality system of claim 7 , wherein the first wave of nodes are marked when the stress is radially distributed to the first wave of nodes.

11. The augmented or virtual reality system of claim 1 , further comprising:
a first positioning system that receives first positioning data of a first augmented or virtual reality system, the first positioning data indicative of a first location of the first augmented or virtual reality system in the real world; and
a set of pose tagged images comprising pose information of the set of augmented or virtual reality system and corresponding to the plurality of existing keyframes.

12. The augmented or virtual reality system of claim 11 , further comprising:
the processor further configured to generate or update first finer granular positioning data corresponding to the first location for the simplified map at least by presenting only respective fingerprints of two physical spaces represented as two corresponding point nodes in the simplified map and a relationship between the two physical spaces.

13. The augmented or virtual reality system of claim 12 , wherein the first positioning data is less granular or precise than the first finer granular positioning data, and the first finer granular positioning data is not achievable through the first positioning data that is provided by the first positioning system and includes first coordinate information of the first location in the real world.

14. The augmented or virtual reality system of claim 13 , further comprising:
the processor further configured to identify parametric geometry data comprising raster imagery, point clouds, descriptor clouds, or geometric definitions of features in the real world; and
a passable world model that comprises the parametric geometry data, wherein a smaller portion of the passable world model is transmitted from a remote computing system through a computer network to the first augmented or virtual reality system by using the simplified map based in part or in whole upon information captured by the first augmented or virtual reality system and transmitted to the remote computing system.

15. The augmented or virtual reality system of claim 14 , further comprising:
a second augmented or virtual reality system in the set of augmented or virtual reality systems;
a second positioning system that receives second positioning data of the second augmented or virtual reality system, the second positioning data indicative of a second location of the second augmented or virtual reality system in the real world;
the processor further configured to generate or update second finer granular positioning data corresponding to the second location for the simplified map;
the first augmented or virtual reality system configured to transmit at least a part of the passable world model to the second augmented or virtual reality system; and
the second augmented or virtual reality system configured to receive the at least the part of the passable world model from the first augmented or virtual reality system.

16. The augmented or virtual reality system of claim 15 , wherein the second augmented or virtual reality system is first positioned in the real world by using the simplified map before the parametric geometry data including the geometric definitions associated with the at least the part of the passable world model is transmitted to the second augmented or virtual reality system.

17. The augmented or virtual reality system of claim 1 , wherein an edge connecting two point nodes in the simplified map represents a shared augmented or virtual reality system or a shared user between the two point nodes.

18. The augmented or virtual reality system of claim 17 , wherein an edge connecting two point nodes in the simplified map does not represent geographical connectivity in the real world between the two point nodes.

19. The augmented or virtual reality system of claim 1 , further comprising:
the processor further configured to identify loop-closure stresses in the geometric map or geometric configurations of one or more physical spaces in the real world based in part or in whole upon the simplified map

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A

Classifications

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/304

  For headphones · CPC title

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• A63F13/577

  using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars · CPC title