Artificial realty augments and surfaces

Patent metadata

Field	Value
Publication number	US-11651573-B2
Application number	US-202117499233-A
Country	US
Kind code	B2
Filing date	Oct 12, 2021
Priority date	Aug 31, 2020
Publication date	May 16, 2023
Grant date	May 16, 2023

Abstract

Aspects of the present disclosure are directed to providing an artificial reality environment with augments and surfaces. An “augment” is a virtual container in 3D space that can include presentation data, context, and logic. An artificial reality system can use augments as the fundamental building block for displaying 2D and 3D models in the artificial reality environment. For example, augments can represent people, places, and things in an artificial reality environment and can respond to a context such as a current display mode, time of day, a type of surface the augment is on, a relationship to other augments, etc. Augments can be on a “surface” that has a layout and properties that cause augments on that surface to display in different ways. Augments and other objects (real or virtual) can also interact, where these interactions can be controlled by rules for the objects evaluated based on information from the shell.

First claim

We claim:

1. A method for orchestrating contextual virtual container display updates for an artificial reality environment, the method comprising:
receiving multiple requests for virtual containers, wherein each request of the multiple requests is associated with a manifest specifying A) one or more parameters for a virtual container, of the virtual containers, corresponding to the request and B) at least two display modes;
registering each of the virtual containers for context factor notifications, wherein the registration for each particular virtual container includes:
analyzing conditions for one or more of the display modes, defined in the manifest corresponding to the particular virtual container, to determine one or more context factors checked when enabling the one or more display modes for the particular virtual container; and
associating, in a mapping for context factor notifications, the particular virtual container with the one or more context factors;

receiving artificial reality environment and sensor data and determining a corresponding changed context factor;
identifying, based on the mapping for context factor notifications, a specific virtual container registered for the changed context factor;
notifying the specific virtual container of the changed context factor, wherein, in response to the notifying, the specific virtual container:
invokes logic, for a particular display mode of the one or more display modes of that specific virtual container, that has conditions that match the changed context factor; and
enables the particular display mode in response to the invoked logic.

2. The method of claim 1 , wherein the invoking the logic includes determining that the particular display mode is a deepest display mode, in a hierarchy of display modes for the specific virtual container, with conditions that cause the particular display mode to be enabled.

3. The method of claim 1 , wherein enabling the particular display mode includes preventing the specific virtual container from outputting part of its current presentation data by clipping output that is outside a boundary area.

4. The method of claim 1 , wherein enabling the particular display mode includes moving the specific virtual container to a side of the user's field of view.

5. The method of claim 1 , wherein the one or more parameters, for the virtual container corresponding to the request, include one or more of: a virtual container type, a virtual container shape, a spatial orientation, location types eligible for the placement of the virtual container, or any combination thereof.

6. The method of claim 1 , wherein at least one of the multiple requests for virtual containers was generated as a result of a user performing an interaction with a content item, that is in another previously-created virtual container, wherein the interaction was previously defined to signal creating a new virtual container based on the content item.

7. The method of claim 1 ,
wherein the determining the changed context factor includes identifying a changed mode of an artificial reality system; and
wherein the enabling the particular display mode causes the virtual container to be set to a maximum size and to be moved to a particular location corresponding to the current mode of the artificial reality system.

8. The method of claim 1 ,
wherein the particular display mode, of the specific virtual container, is an extension of an other display mode of the specific virtual container; and
wherein a condition for enabling the particular display mode is that a condition, associated with the other display mode, evaluates to true.

9. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for orchestrating contextual virtual container display updates for an artificial reality environment, the process comprising:
registering one or more virtual containers, each having at least two display modes, for context factor notifications, wherein the registration for each particular virtual container includes:
analyzing conditions for one or more of the display modes to determine one or more context factors checked when enabling the one or more display modes for the particular virtual container; and
defining associations between the particular virtual container and the one or more context factors;

identifying, based on the associations, a specific virtual container registered for an identified changed context factor; and
notifying the specific virtual container of the changed context factor;
wherein, in response to evaluating a condition for a particular display mode based on the notification of the changed context factor, the specific virtual container enables the particular display mode.

10. The non-transitory computer-readable storage medium of claim 9 ,
wherein the identifying the changed context factor includes identifying a changed mode of an artificial reality system; and
wherein the enabling the particular display mode causes the virtual container to be set to a maximum size and/or to be moved to a particular location corresponding to the current mode of the artificial reality system.

11. The non-transitory computer-readable storage medium of claim 9 ,
wherein the particular display mode, of the specific virtual container, is an extension of an other display mode of the specific virtual container; and
wherein the condition for the particular display mode is that a condition, associated with the other display mode, evaluates to true.

12. The non-transitory computer-readable storage medium of claim 9 , wherein the evaluating the condition includes determining that the particular display mode is a deepest display mode, in a hierarchy of display modes for the specific virtual container, with conditions that cause the particular display mode to be enabled.

13. The non-transitory computer-readable storage medium of claim 9 , wherein enabling the particular display mode includes preventing the specific virtual container from outputting part of its current presentation data by clipping output that is outside a boundary area.

14. The non-transitory computer-readable storage medium of claim 9 , wherein enabling the particular display mode includes moving the specific virtual container to a side of the user's field of view.

15. The non-transitory computer-readable storage medium of claim 9 , wherein the registering the one or more virtual containers is a result of a user performing an interaction with a content item, that is in another previously-created virtual container, wherein the interaction was previously defined to signal creating a new virtual container based on the content item.

16. A computing system for orchestrating contextual virtual container display updates for an artificial reality environment, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
registering one or more virtual containers, each having at least two display modes, for context factor notifications, wherein the registration for each particular virtual container includes:
analyzing conditions for one or more of the display modes to determine one or more context factors checked when enabling the one or more display modes for the particular virtual container; and
defining associations between the particular virtual container and the one or more context factors;

identifying, based on the association

Assignees

• Meta Platforms Tech Llc

Inventors

• Tichenor James
• Zwiegincew Arthur
• Schoen Hayden
• Marcolina Alex
• Alt Gregory
• Harris Todd
• Deng Merlyn
• Fox Barrett
• Hlavac Michal

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title