Triangulation of points using known points in augmented or virtual reality systems

Patent metadata

Field	Value
Publication number	US-9766703-B2
Application number	US-201514706996-A
Country	US
Kind code	B2
Filing date	May 8, 2015
Priority date	Apr 18, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

An augmented reality display system comprises a passable world model data comprises a set of map points corresponding to one or more objects of the real world. The augmented reality system also comprises a processor to communicate with one or more individual augmented reality display systems to pass a portion of the passable world model data to the one or more individual augmented reality display systems, wherein the piece of the passable world model data is passed based at least in part on respective locations corresponding to the one or more individual augmented reality display systems.

First claim

What is claimed is:

1. A method of displaying augmented reality, comprising:
capturing a set of 2D points of a real-world environment at one or more image capturing devices of one or more augmented reality systems;
determining respective positions of the one or more image capturing devices for the set of 2D points;
determining a three-dimensional (3D) position of a 2D point of the set of 2D points based at least in part on the set of 2D points and the respective positions; and
generating, at an augmented reality system, a virtual content for the real-world environment by using at least the three-dimensional position that is shared via a computer network.

2. The method of claim 1 , further comprising determining one or more respective poses of the one or more augmented reality systems.

3. The method of claim 2 , wherein a pose of the one or more respective poses comprises a position or an orientation of an augmented reality system in relation to at least one 2D point in the set of 2D points.

4. The method of claim 1 , further comprising:
segmenting one or more 2D images of the real-world environment into 2D segmented image features;
identifying one or more 3D nodes represented by respective 3D positions determined from the set of 2D points; and
determining a correlation between the segmented 2D segmented image features with the one or more 3D nodes.

5. The method of claim 1 , further comprising determining respective depth information of one or more objects through at least one of the one or more augmented reality systems.

6. The method of claim 1 , determining a pose of a separate augmented reality system based at least in part on the 3D position of the point.

7. The method of claim 4 , further comprising;
deriving one or more object structures or one or more properties for one or more objects in the real-world environment based in part or in whole upon the correlation; and
constructing respective geometries of the one or more objects for the virtual content.

8. The method of claim 1 , determining the three-dimensional position of the 2D point further comprising:
identifying a set of keyframes comprising the set of 2D points; and
identifying a set of keyframe positions respectively corresponding to the set of keyframes.

9. The method of claim 8 , further comprising:
projecting a plurality of geometric entities from the set of keyframe positions of the set of keyframes to the set of 2D points; and
identifying an intersection between two geometric entities of the plurality of geometric entities as the three-dimensional position of the 2D point.

10. The method of claim 9 , further comprising:
determining a relative likelihood of having a physical object at the 3D position based in part or in whole upon pixel coordinates of the 2D point.

11. The method of claim 9 , projecting the plurality of geometric entities further comprising:
projecting a triangular cone from a keyframe position of the set of keyframe positions to the 2D point of the set of 2D points at least by bisecting the triangular cone with a line segment connecting the keyframe position and the 2D point.

12. The method of claim 11 , further comprising:
shading an interior of triangular cone so that the line segment bisecting the triangular cone is rendered brighter.

13. The method of claim 12 , further comprising:
identifying a resolution characteristic and lens calibration characteristic of an image capturing device of the one or more image capturing devices; and
configuring a buffer an image capturing device into a summing buffer so that a brighter spot represents a candidate location of a candidate 2D point.

14. The method of claim 9 , further comprising:
reducing computational resource utilization at least by performing a process, the process comprising:
projecting the plurality of geometric entities from a number of unmatched 2D points in a number of prior keyframes into a first texture of a new keyframe; and
encoding a keyframe identifier and a feature identifier for the new keyframe.

15. The method of claim 14 , further comprising:
identifying one or more 2D points from the new keyframe;
generating a second texture from the one or more 2D points from the new keyframe; and
masking the first texture with the second texture.

16. An apparatus for rendering augmented reality virtual contents, comprising:
one or more augmented reality systems comprising one or more image capturing devices, the one or more image capturing devices configured to capture a set of 2D points of a real-world environment;
a remote computing system configured to receive the set of 2D points, to determine respective positions of the one or more image capturing devices for the set of 2D points, to determine a three-dimensional (3D) position of a 2D point of the set of 2D points based at least in part on the set of 2D points and the respective positions, and to generate, at an augmented reality system, a virtual content for the real-world environment by using at least the three-dimensional position that is shared via a computer network.

17. The apparatus of claim 16 , further comprising:
the remote computing system further configured to segment one or more 2D images of the real-world environment into 2D segmented image features, to identify one or more 3D nodes represented by respective 3D positions determined from the set of 2D points, and to determine a correlation between the segmented 2D segmented image features with the one or more 3D nodes.

18. The apparatus of claim 17 , comprising:
the remote computing system further configured to derive one or more object structures or one or more properties for one or more objects in the real-world environment based in part or in whole upon the correlation, and to construct respective geometries of the one or more objects for the virtual content.

19. The apparatus of claim 16 , further comprising:
the remote computing system further configured to identify a set of keyframes comprising the set of 2D points, to identify a set of keyframe positions respectively corresponding to the set of keyframes, to project a plurality of geometric entities from the set of keyframe positions of the set of keyframes to the set of 2D points, and to identify an intersection between two geometric entities of the plurality of geometric entities as the three-dimensional position of the 2D point.

20. The apparatus of claim 19 , further comprising:
the remote computing system further configured to reduce computational resource utilization at least by projecting the plurality of geometric entities from a number of unmatched 2D points in a number of prior keyframes into a first texture of a new keyframe, identifying one or more 2D points from the new keyframe, generating a second texture from the one or more 2D points from the new keyframe, and masking the first texture with the second texture.

Assignees

• Magic Leap Inc

Inventors

• Miller Samuel A

Classifications

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/304

  For headphones · CPC title

• G06Q30/02

  Marketing; Price estimation or determination; Fundraising · CPC title

• A63F13/577

  using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars · CPC title