Matching content to a spatial 3D environment

Patent metadata

Field	Value
Publication number	US-11373376-B2
Application number	US-202117142210-A
Country	US
Kind code	B2
Filing date	Jan 5, 2021
Priority date	May 1, 2017
Publication date	Jun 28, 2022
Grant date	Jun 28, 2022

Abstract

Systems and methods for matching content elements to surfaces in a spatially organized 3D environment. The method includes receiving content, identifying one or more elements in the content, determining one or more surfaces, matching the one or more elements to the one or more surfaces, and displaying the one or more elements as virtual content onto the one or more surfaces.

First claim

What is claimed is:

1. A method for matching content to a plurality of surfaces of environment of the user, the method comprising:
identifying a content element having a plurality of different attributes corresponding to the plurality of different attributes of each of the plurality of surfaces;
determining a plurality of different attributes of each of the plurality of surfaces respectively corresponding to the plurality of different attributes of the content element;
respectively comparing the plurality of different attributes of the content element to the plurality of different attributes of each of the plurality of surfaces;
calculating a plurality of scores for the respective plurality of surfaces based on the respective comparisons;
selecting a surface having the highest score from the plurality of surfaces;
storing a mapping of the content element to the selected surface; and
displaying the content element on the selected surface to the user.

2. The method of claim 1 , wherein the identified content element is a 3D content element.

3. The method of claim 1 , wherein the plurality of different attributes of the content element are weighted differently.

4. The method of claim 1 , wherein the plurality of different attributes of the content element comprise a dot product orientation surface relationship, a texture, and a color.

5. The method of claim 1 , wherein the surface on which the content element is displayed to the user is the selected surface.

6. The method of claim 1 , further comprising comparing the highest score to a threshold score, displaying the content element on either the selected surface or a virtual surface based on the comparison.

7. The method of claim 6 , wherein the content element is displayed on the selected surface if the threshold score is greater than the threshold score, and displaying the content element on the virtual surface if the threshold score is less than the threshold score.

8. The method of claim 1 , further comprising overriding the selected surface, and selecting another surface, wherein the surface on which the content element is displayed to the user is the other surface.

9. The method of claim 1 , further comprising moving the displayed content element from the surface to another surface.

10. The method of claim 1 , wherein the displayed content element is moved from the surface to the other surface via a hand gesture of the user.

11. An augmented reality (AR) display system, comprising:
a head-mounted system comprising:
one or more sensors, and
one or more cameras comprising outward facing cameras;
a processor to execute a set of program code instructions; and
a memory to hold the set of program code instructions, in which the set of program code instructions comprises program code to perform:
identifying a content element having a plurality of different attributes corresponding to the plurality of different attributes of each of the plurality of surfaces;
determining a plurality of different attributes of each of the plurality of surfaces respectively corresponding to the plurality of different attributes of the content element;
respectively comparing the plurality of different attributes of the content element to the plurality of different attributes of each of the plurality of surfaces;
calculating a plurality of scores for the respective plurality of surfaces based on the respective comparisons;
selecting a surface having the highest score from the plurality of surfaces;
storing a mapping of the content element to the selected surface; and
displaying the content element on the selected surface to the user.

12. The system of claim 11 , wherein the identified content element is a 3D content element.

13. The system of claim 11 , wherein the plurality of different attributes of the content element are weighted differently.

14. The system of claim 11 , wherein the plurality of different attributes of the content element comprise a dot product orientation surface relationship, a texture, and a color.

15. The system of claim 11 , wherein the surface on which the content element is displayed to the user is the selected surface.

16. The system of claim 11 , wherein the program code further performs comparing the highest score to a threshold score, displaying the content element on either the selected surface or a virtual surface based on the comparison.

17. The system of claim 16 , wherein the content element is displayed on the selected surface if the threshold score is greater than the threshold score, and displaying the content element on the virtual surface if the threshold score is less than the threshold score.

18. The system of claim 11 , wherein the program code further preforms overriding the selected surface, and selecting another surface, wherein the surface on which the content element is displayed to the user is the other surface.

19. The system of claim 11 , wherein the program code further preforms moving the displayed content element from the surface to another surface.

20. The system of claim 11 , wherein the programmed code further allows the displayed content element to be moved from the surface to the other surface via a hand gesture of the user.

Assignees

• Magic Leap Inc

Inventors

• Bastov Denys
• Ng-Thow-Hing Victor
• Reinhardt Benjamin Zaaron
• ZOLOTAREV Leonid
• Pellet Yannick
• Marchenko Aleksei
• Meaney Brian Everett
• Shelton Marc Coleman
• Geiman Megan Ann
• Gotcher John A
• Bogue Matthew Schon
• Balasubramanyam Shivakumar
• Ruediger Jeffrey Edward
• Lundmark David Charles

Classifications

• G02B30/00

  Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images (in microscopes G02B21/22) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title