Artificial reality augments and surfaces

Patent metadata

Field	Value
Publication number	US-11769304-B2
Application number	US-202117522326-A
Country	US
Kind code	B2
Filing date	Nov 9, 2021
Priority date	Aug 31, 2020
Publication date	Sep 26, 2023
Grant date	Sep 26, 2023

Abstract

Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

First claim

We claim:

1. A method for managing multiple virtual containers in an artificial reality environment, the method comprising:
receiving an identification of a surface in the artificial reality environment with at least a surface type property;
identifying at least one virtual container, of the multiple virtual containers, that is associated with the identified surface; and
providing one or more properties of the identified surface to the identified at least one virtual container;
wherein the identified at least one virtual container selects a current display mode based at least in part on the one or more properties of the identified surface, and
wherein the current display mode A) controls how the identified at least one virtual container writes content into a defined size or shape of an area or a volume that was set for the identified at least one virtual container in the current display mode and B) corresponds to the surface type property of the identified surface.

2. The method of claim 1 ,
wherein the surface type property of the identified surface specifies a spatial orientation of the identified surface; and
wherein the identified at least one virtual container evaluates one or more conditions to select the current display mode that corresponds to the spatial orientation of the identified surface.

3. The method of claim 1 , wherein the surface type property of the identified surface specifies an orientation of the identified surface or a type of object the identified surface is attached to.

4. The method of claim 1 , wherein the surface type property of the identified surface specifies types of virtual containers that can be added to the identified surface.

5. The method of claim 1 , wherein the one or more properties of the identified surface further include tags identified by machine learning models trained to specify tags for particular surfaces based on an identified context of each particular surface including at least one of surface size, surface position, real-world or virtual objects associated with the particular surface, or any combination thereof.

6. The method of claim 1 ,
wherein a layout specified for the identified surface defines how a plurality of virtual containers added to the identified surface can be placed by defining slots on the identified surface; and
wherein the identified at least one virtual container, when added to the identified surface, is assigned one of the slots, causing a location of the identified at least one virtual container to be set according to a location of the assigned slot.

7. The method of claim 1 ,
wherein a layout specified for the identified surface defines how a plurality of virtual containers, when added to the identified surface, can be placed by the layout defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned one of the slots, causing a location of the identified at least one virtual container to be set according to a location of the assigned slot; and
wherein the layout for the identified surface specifies one of:
a list layout wherein the plurality of virtual containers added to the identified surface are placed in a horizontal line spaced uniformly from each other;
a stack layout wherein the plurality of virtual containers added to the identified surface are placed in a vertical line spaced uniformly from each other; or
a grid layout wherein the plurality of virtual containers added to the identified surface are placed on a grid with a number of grid slots in each dimension of the identified surface, specified based on a number of the plurality of virtual containers added to the identified surface.

8. The method of claim 1 ,
wherein a layout specified for the identified surface defines how a plurality of virtual containers, when added to the identified surface, can be placed by dynamically defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned one of the slots, causing a location of the identified at least one virtual container to be set according to a location of the assigned slot; and
wherein the layout for the identified surface specifies a freeform layout wherein the assigned slot is created on the identified surface, for the identified at least one virtual container, according to where the identified at least one virtual container was placed on the identified surface.

9. The method of claim 1 ,
wherein a layout specified for the identified surface defines how a plurality of virtual containers, when added to the identified surface, can be placed by dynamically defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned one of the slots, causing a location of the identified at least one virtual container to be set according to a location of the assigned slot; and
wherein the layout for the identified surface is dynamic such that a number, size, and/or position of slots in the layout are specified in response to a number of the plurality of virtual containers, when added to the identified surface, a size of the plurality of virtual containers, when added to the identified surface, and/or where the plurality of virtual containers were initially placed on the identified surface.

10. The method of claim 1 , wherein the identified surface is one or more of:
a surface positioned relative to an artificial reality system that controls the artificial reality environment;
a surface positioned relative to a real-world object detected by a machine learning recognizer trained to recognize one or more particular types of objects; or
a surface positioned relative to a real-world surface determined to have at least minimum geometric features.

11. The method of claim 1 , wherein the identified at least one virtual container was associated with the identified surface in response to one of:
a user performing an interaction to add the identified at least one virtual container to the identified surface;
the identified at least one virtual container having been created with an association to the identified surface based on a particular virtual container, that caused creation of the identified at least one virtual container, being on the identified surface; or
execution of logic of the identified at least one virtual container or enablement of a display mode of the identified at least one virtual container, in response to the identified at least one virtual container having receiving context factors, that caused the identified at least one virtual container to be added to the identified surface.

12. A computing system for managing multiple virtual containers in an artificial reality environment, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
selecting, by at least one virtual container, a current display mode based at least in part on one or more properties of a surface,
wherein the one or more properties are received from an artificial reality environment controlling application that:
receives an identification of the surface in the artificial reality environment with at least a surface type property;
identifies the at least one virtual container, of the multiple virtual containers, that is associated with the identified surface; and
provides the one or more properties of the identified surface to the identified at least one virtual container; and

wherein the cu

Assignees

• Meta Platforms Tech Llc

Inventors

• Tichenor James
• Zwiegincew Arthur
• Schoen Hayden
• Marcolina Alex
• Alt Gregory
• Harris Todd
• Deng Merlyn
• Fox Barrett
• Hlavac Michal

Classifications

• G06Q10/48

  using social graphs · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title