Rendering techniques to find new map points in augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-10043312-B2
Application number	US-201514705994-A
Country	US
Kind code	B2
Filing date	May 7, 2015
Priority date	Apr 18, 2014
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

An augmented reality system comprises one or more sensors to capture a set of map points pertaining to the real world, wherein the set of map points are captured through a plurality of augmented reality systems, and a processor to determine a position of a plurality of keyframes that captured the set of map points, and to determine a set of new map points based at least in part on the captured set of map points and the determined position of the plurality of keyframes.

First claim

What is claimed is:

1. An augmented reality system, comprising:
one or more sensors to capture a set of map points pertaining to the real world, wherein the set of map points are captured through a plurality of augmented reality systems; and
a processor to determine a position of a plurality of keyframes that captured the set of map points, render lines from the determined positions of the plurality of keyframes to respective map points captured from the plurality of keyframes, identify points of intersection between the rendered lines, and to determine a set of new map points based at least in part on the identified points of intersection.

2. The augmented reality system of claim 1 , wherein the set of new map points are determined based at least in part on a pixel pitch corresponding to the identified points of intersection.

3. The augmented reality system of claim 1 , wherein the set of new map points are added to a map of the real world.

4. The augmented reality system of claim 3 , wherein the virtual content is delivered to one or more augmented reality display systems based at least in part on the map of the real world.

5. An augmented reality system, comprising:
one or more sensors to capture a set of map points pertaining to the real world, wherein the set of map points are captured through a plurality of augmented reality systems; and
a processor to determine a position of a plurality of keyframes that captured the set of map points, render triangular cones from the determined positions of the plurality of keyframes to respective map points captured from the plurality of keyframes, wherein the captured map points lie on bisectors of the triangular cones and determine a set of new map points based at least in part on the rendered triangular cones.

6. The augmented reality system of claim 5 , wherein the processor selectively shades the triangular cones such that the bisectors of the triangular cones are the brightest portions of the triangular cones.

7. The augmented reality system of claim 6 , wherein the processor identifies points of intersection between at least two rendered triangular cones, wherein the set of new map points are based at least in part on the identified points of intersection.

8. The augmented reality system of claim 7 , wherein the set of new map points are determined based at least in part on the brightness of the identified points of intersection.

9. The augmented reality system of claim 7 , wherein the set of new map points are determined based at least in part on a pixel pitch corresponding to the identified points of intersection.

10. The augmented reality system of claim 5 , wherein the set of new map points are added to a map of the real world.

11. The augmented reality system of claim 10 , wherein the virtual content is delivered to one or more augmented reality display systems based at least in part on the map of the real world.

12. An augmented reality system, comprising:
one or more sensors to capture a set of map points pertaining to the real world, wherein the set of map points are captured through a plurality of augmented reality systems; and
a processor to determine a position of a plurality of keyframes that captured the set of map points, place a virtual keyframe in relation to an existing set of keyframes, and determine a set of new map points based at least in part on the virtual keyframe, wherein the processor determines a most orthogonal direction to the existing set of keyframes, and positions the virtual keyframe at the determined orthogonal direction.

13. The augmented reality system of claim 12 , wherein the most orthogonal direction is determined along one of an x coordinate, a y coordinate, and a z coordinate.

14. The augmented reality system of claim 12 , wherein the processor renders lines from the virtual keyframe to the set of map points, and determines the new map points based at least in part on one or more points of intersection of the rendered lines.

15. The augmented reality system of claim 14 , wherein the processor applies a summing buffer to determine the points of intersection.

16. The augmented reality system of claim 12 , wherein the processor renders triangular cones from the virtual keyframe to the set of map points, and determines the new map points based at least in part on one or more points of intersection.

17. The augmented reality system of claim 16 , wherein the processor performs a bundle adjust to correct a location of a new map point of the set of new map points.

18. The augmented reality system of 11 , wherein the set of new map points are added to a map of the real world.

19. The augmented reality system of claim 18 , wherein virtual content is delivered to one or more augmented reality display systems based at least in part on the map of the real world.

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A
• Hand Randall E

Classifications

• A63F13/577

  using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• A63F13/57

  Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game (A63F13/56 takes precedence) · CPC title

• G02B27/0179

  Display position adjusting means not related to the information to be displayed · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title