Artificial reality environment control through an artificial reality environment schema

Patent metadata

Field	Value
Publication number	US-12374050-B2
Application number	US-202418652220-A
Country	US
Kind code	B2
Filing date	May 1, 2024
Priority date	Jul 7, 2021
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

An artificial reality environment (XRE) schema is defined that supports controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

First claim

We claim:

1. A method for device-agnostic representation of an artificial reality environment (XRE) using an XRE schema, the method comprising:
defining multiple features, in the artificial reality environment, each as an element defined by the XRE schema,
wherein at least two of the elements each has a defined name and an element list of other elements that element includes, and
wherein at least one element comprises one or more quantity guarantees with respect to its element list;
performing, for two or more elements defined by the XRE schema, application programming interface element constructor calls,
wherein constructed instances for the two or more elements are returned in response to the element constructor calls, and
wherein at least one of the called element constructors, for one element of the two or more elements, enforces the one or more quantity guarantees with respect the element list for the one element; and

adding a node, for each of the constructed instances of each element, to a hierarchy for the artificial reality environment,
wherein each edge in the hierarchy specifies a parent/child link between elements where the parent element contains the child element; and
wherein the method is performed on a first type of artificial reality device and at least one of the elements is accessed by a second type of artificial reality device with a type different from the first type of artificial reality device.

2. The method of claim 1 , wherein each element is either an entity-type element or a component-type element.

3. The method of claim 2 , wherein, in the hierarchy, each entity only links to components and each component only links to entities.

4. The method of claim 2 ,
wherein the at least one element comprises two elements including a first entity and a first component,
wherein the first entity specifies multiple component quantity guarantees, each component quantity guarantee defining an amount of components of a particular type the first entity is linked to; and
wherein the first component specifies at least one entity quantity guarantee, each entity quantity guarantee defining an amount of entities of a particular type the first component is linked to.

5. The method of claim 1 , wherein a root node of the hierarchy is a room element specifying an area that is traversable without going through a doorway, taking stairs, or taking an elevator.

6. The method of claim 1 , wherein at least one element in the hierarchy is linked to an anchor element specifying where, in 3D space, the least one element exists in relation to an origin point.

7. The method of claim 1 ,
wherein the hierarchy is a first hierarchy containing a first root node, and
wherein the first root node includes an element that defines a connection to a second root node of a second hierarchy for another artificial reality environment.

8. The method of claim 1 , wherein performing the application programming interface element constructor calls further comprises:
obtaining element parameters corresponding to one or more element constructors defined for the elements,
wherein, for a particular one of the elements, one or more of the element parameters were generated by machine perception systems that automatically identified aspects of the real world around the artificial reality device of the first type; and

calling the one or more element constructors.

9. The method of claim 1 , wherein at least one particular element, of the elements, is acted upon by:
obtaining a handle for the at least one particular element; and
via the handle:
accessing a query from a query list of the at least one particular element; or
setting or getting an element part of the at least one particular element.

10. The method of claim 1 , wherein the first type of artificial reality device is a virtual reality device and the second type of artificial reality device is an augmented reality or mixed reality device.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for device-agnostic representation of an artificial reality environment (XRE) using an XRE schema, the process comprising:
defining multiple features, in the artificial reality environment, each as an element defined by the XRE schema,
wherein at least two of the elements each has a defined name and an element list of other elements that element includes, and
wherein at least one element comprises one or more quantity guarantees with respect to its element list;
performing, for two or more elements defined by the XRE schema, application programming interface element constructor calls,
wherein constructed instances for the two or more elements are returned in response to the element constructor calls, and
wherein at least one of the called element constructors, for one element of the two or more elements, enforces the one or more quantity guarantees with respect the element list for the one element; and

adding a node, for each of the constructed instances of each element, to a hierarchy for the artificial reality environment,
wherein each edge in the hierarchy specifies a parent/child link between elements where the parent element contains the child element.

12. The non-transitory computer-readable storage medium of claim 11 , wherein each element is either an entity-type element or a component-type element.

13. The non-transitory computer-readable storage medium of claim 12 , wherein, in the hierarchy, each entity only links to components and each component only links to entities.

14. The non-transitory computer-readable storage medium of claim 12 ,
wherein the at least one element comprises two elements including a first entity and a first component,
wherein the first entity specifies multiple component quantity guarantees, each component quantity guarantee defining an amount of components of a particular type the first entity is linked to; and
wherein the first component specifies at least one entity quantity guarantee, each entity quantity guarantee defining an amount of entities of a particular type the first component is linked to.

15. The non-transitory computer-readable storage medium of claim 11 , wherein a root node of the hierarchy is a room element specifying an area that is traversable without going through a doorway, taking stairs, or taking an elevator.

16. The non-transitory computer-readable storage medium of claim 11 , wherein at least one element in the hierarchy is linked to an anchor element specifying where, in 3D space, the least one element exists in relation to an origin point.

17. A computing system for device-agnostic representation of an artificial reality environment (XRE) using an XRE schema, the computing system comprising:
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
defining multiple features, in the artificial reality environment, each as an element defined by the XRE schema,
wherein at least two of the elements each has a defined name and an element list of other elements that element includes, and
wherein at least one element comprises one or more quantity guarantees with respect to its element list;

performing, for two or more elements defined by the XRE schema, application programming interface element constructor calls,
wherein constructed instances for the two or more elements are returned in response to the element constructor ca

Assignees

• Meta Platforms Tech Llc

Inventors

• Noris Gioacchino
• Hlavac Michal
• Furgale Paul Timothy
• Schmid Johannes Joachim
• Mohan Anush
• Tanner Christopher Richard

Classifications

• G06F16/909

  using geographical or spatial information, e.g. location (spatiotemporally dependent retrieval from the web G06F16/9537) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F8/24

  Object-oriented · CPC title

• G06T19/00 Primary

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title