Artificial reality augments and surfaces

Patent metadata

Field	Value
Publication number	US-11227445-B1
Application number	US-202017008487-A
Country	US
Kind code	B1
Filing date	Aug 31, 2020
Priority date	Aug 31, 2020
Publication date	Jan 18, 2022
Grant date	Jan 18, 2022

Abstract

Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

First claim

We claim:

1. A method for managing multiple virtual containers in an artificial reality environment, the method comprising:
receiving an identification of a surface in the artificial reality environment, wherein the identified surface has properties including a location in the artificial reality environment, a surface type property, and a layout defining how a plurality of virtual containers can be placed when added to the identified surface;
identifying at least one virtual container, of the multiple virtual containers, that is associated with the identified surface; and providing one or more of the properties of the identified surface to the identified at least one virtual container;
wherein the identified at least one virtual container is configured to evaluate, based on at least one of the properties of the identified surface, one or more conditions associated with display modes for the identified at least one virtual container, to select a current display mode for presenting the identified at least one virtual container in association with the identified surface, and
wherein the current display mode, of the display modes A) controls how the identified at least one virtual container writes content into a defined size or shape of an area or a volume set for the identified at least one virtual container in the current display mode and B) corresponds to the surface type property of the identified surface.

2. The method of claim 1 ,
wherein the surface type property of the identified surface specifies a spatial orientation of the identified surface; and
wherein the identified at least one virtual container evaluates at least one of the one or more conditions to select the current display mode that corresponds to the spatial orientation of the identified surface.

3. The method of claim 1 , wherein the surface type property of the identified surface specifies an orientation of the identified surface or a type of object the identified surface is attached to.

4. The method of claim 1 , wherein the surface type property of the identified surface specifies types of virtual containers that can be added to the identified surface.

5. The method of claim 1 , wherein the properties for the identified surface further include tags identified by machine learning models trained to specify one or more of the tags for particular surfaces based on an identified context of each particular surface including at least one of surface size, surface position, real-world or virtual objects associated with that particular surface, or any combination thereof.

6. The method of claim 1 ,
wherein the layout defines how the plurality of virtual containers added to the identified surface can be placed by defining slots on the identified surface; and
wherein the identified at least one virtual container, when added to the identified surface, is assigned a slot, causing a location of the identified at least one virtual container to be set according to a location of the slot.

7. The method of claim 1 ,
wherein the layout defines how the plurality of virtual containers, when added to the identified surface, can be placed by defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned a slot, causing a location of the identified at least one virtual container to be set according to a location of the slot; and
wherein the layout for the identified surface specifies one of:
a list layout wherein the plurality of virtual containers added to the identified surface are placed in a horizontal line spaced uniformly from each other;
a stack layout wherein the plurality of virtual containers added to the identified surface are placed in a vertical line spaced uniformly from each other; or
a grid layout wherein the plurality of virtual containers added to the identified surface are placed on a grid with a number of grid slots in each dimension of the identified surface specified based on a number of the plurality of virtual containers added to the identified surface.

8. The method of claim 1 ,
wherein the layout defines how the plurality of virtual containers, when added to the identified surface, can be placed by dynamically defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned a slot, causing a location of the identified at least one virtual container to be set according to a location of the slot; and
wherein the layout for the identified surface specifies a freeform layout wherein the slot is created on the identified surface, for the identified at least one virtual container, according to where the identified at least one virtual container was placed on the identified surface.

9. The method of claim 1 ,
wherein the layout defines how the plurality of virtual containers, when added to the identified surface, can be placed by dynamically defining slots on the identified surface;
wherein the identified at least one virtual container, when added to the identified surface, is assigned a slot, causing a location of the identified at least one virtual container to be set according to a location of the slot; and
wherein the layout for the identified surface is dynamic such that a number, size, and/or position of slots in the layout are specified in response to a number of the plurality of virtual containers, when added to the identified surface, a size of the plurality of virtual containers, when added to the identified surface, and/or where the plurality of virtual containers were initially placed on the identified surface.

10. The method of claim 1 , wherein the identified surface is one or more of:
a surface positioned relative to an artificial reality system that controls the artificial reality environment;
a surface positioned relative to a real-world object detected by a machine learning recognizer trained to recognize one or more particular types of objects; or
a surface positioned relative to a real-world surface determined to have at least minimum geometric features.

11. The method of claim 1 , wherein the identified at least one virtual container was associated with the identified surface in response to one of:
a user performing an interaction to add the identified at least one virtual container to the identified surface;
the identified at least one virtual container having been created with an association to the identified surface based on a particular virtual container, that caused creation of the identified at least one virtual container, being on the identified surface; or
execution of logic of the identified at least one virtual container or enablement of a particular display mode of the identified at least one virtual container, in response to the identified at least one virtual container having receiving context factors, that caused the identified at least one virtual container to be added to the identified surface.

12. A computing system for managing multiple virtual containers in an artificial reality environment, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
receiving an identification of a surface in the artificial reality environment, wherein the identified surface has specified properties;
identifying at least one virtual container, of the multiple virtual containers, that is associated with the identified surface; and
providing one or more of the specified properties of the identified surface to the identified at least one virtu

Assignees

• Facebook Tech Llc

Inventors

• Tichenor James
• Zwiegincew Arthur
• Schoen Hayden
• Marcolina Alex
• Alt Gregory
• Harris Todd
• Deng Merlyn
• Fox Barrett
• Hlavac Michal

Classifications

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06Q30/0643

  graphically representing goods, e.g. 3D product representation · CPC title