Matching content to a spatial 3D environment

Patent metadata

Field	Value
Publication number	US-11875466-B2
Application number	US-202217663792-A
Country	US
Kind code	B2
Filing date	May 17, 2022
Priority date	May 1, 2017
Publication date	Jan 16, 2024
Grant date	Jan 16, 2024

Abstract

Systems and methods for matching content elements to surfaces in a spatially organized 3D environment. The method includes receiving content, identifying one or more elements in the content, determining one or more surfaces, matching the one or more elements to the one or more surfaces, and displaying the one or more elements as virtual content onto the one or more surfaces.

First claim

What is claimed is:

1. A method of matching content elements of content to a spatial three-dimensional (3D) environment, comprising:
a content structurization process comprising deconstructing content by parsing a webpage, thereby identifying a plurality of content elements from the webpage;
an environment structurization process comprising parsing an environment, thereby identifying a plurality of surfaces from the environment; and
a compositing process comprising respectively matching the content elements with the plurality of surfaces, and rendering and displaying the content elements respectively onto the matched surfaces.

2. The method of claim 1 , wherein parsing the webpage comprises identifying and storing attributes of the content into a logical structure for each of the content elements.

3. The method of claim 2 , wherein parsing the environment comprises identifying and storing attributes of the environment content into the logical structure for each of the plurality of surfaces.

4. The method of claim 3 , wherein matching the content elements with the surfaces comprises respectively comparing attributes of the content elements with attributes of the surfaces.

5. The method of claim 4 , wherein the content elements are respectively matched to the surfaces based on the content elements and the surfaces sharing similar or opposing attributes.

6. The method of claim 2 , wherein the logical structure comprises one of an ordered array, a hierarchical table, a tree structure, and a logical graph structure.

7. The method of claim 6 , wherein the logical structure comprises a tree structure having a parent node to which a parent content element is assigned, and at least one child node linked to the parent node and to which at least another child content element is assigned, and wherein the parent content element and the at least one child content element are matched to the surfaces based on a proximity of the surfaces to each other.

8. The method of claim 1 , wherein the matched surfaces further comprises at least one virtual surface.

9. The method of claim 1 , wherein rendering the content elements respectively onto the surfaces comprises scaling the content element to fit the respective matched surfaces.

10. The method of claim 1 , wherein at least some of the content elements matched to the surfaces in a first room are matched to surfaces in a second room when the user leaves the first room and enters a second room.

11. The method of claim 1 , wherein the compositing process determines that one of the surfaces is not suitable for one of the content elements, creates a virtual surface, matches the one content element with the virtual surface, and renders the one content element onto the virtual surface.

12. The method of claim 1 , wherein the compositing process further comprising displaying a plurality of candidate surface options to a user for displaying one of the content elements, and matching the one content element to the surface corresponding to one of the displayed candidate surfaces selected by the user.

13. The method of claim 12 , wherein the plurality of candidate surface options are disposed in a field of view of the user, and wherein the compositing process further comprises displaying an additional candidate surface option to the user for displaying the one content element when the field of view of the user changes to a new field of view in which the additional candidate surface option is disposed.

14. The method of claim 13 , further comprising sensing a headpose change between the field of view and new field of view, and wherein the additional candidate surface option is only displayed to the user if the sensed headpose change exceeds a headpose change threshold.

15. The method of claim 12 , wherein the candidate surfaces are simultaneously displayed to the user.

16. The method of claim 12 , wherein the candidate surfaces are successively displayed to the user.

17. The method of claim 12 , wherein the compositing process further comprises displaying a candidate view of the one of the content elements onto the candidate surface options.

18. The method of claim 12 , wherein the compositing process further comprises storing the candidate surface option selected by the user for display of the one content element as a preferred candidate surface option, and prioritizing the preferred candidate surface option when subsequently matching content elements with surfaces.

19. A method of matching content elements of content to a spatial three-dimensional (3D) environment, comprising:
a content structurization process comprising deconstructing content by parsing a double-sided webpage, thereby identifying a plurality of content elements from the webpage;
an environment structurization process comprising parsing an environment, thereby identifying opposite surfaces of a physical structure; and
a compositing process comprising respectively matching two of the content elements with the opposite surfaces of the physical structure, and rendering and displaying the two of the content elements respectively onto the opposite surfaces of the physical structure.

Assignees

• Magic Leap Inc

Inventors

• Bastov Denys
• Ng-Thow-Hing Victor
• Reinhardt Benjamin Zaaron
• ZOLOTAREV Leonid
• Pellet Yannick
• Marchenko Aleksei
• Meaney Brian Everett
• Shelton Marc Coleman
• Geiman Megan Ann
• Gotcher John A
• Bogue Matthew Schon
• Balasubramanyam Shivakumar
• Ruediger Jeffrey Edward
• Lundmark David Charles

Classifications

• G02B30/00

  Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images (in microscopes G02B21/22) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title