Computationally efficient model selection

Patent metadata

Field	Value
Publication number	US-11699270-B2
Application number	US-202217740052-A
Country	US
Kind code	B2
Filing date	May 9, 2022
Priority date	Sep 28, 2018
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

In various implementations, a device surveys a scene and presents, within the scene, a extended reality (XR) environment including one or more assets that evolve over time (e.g., change location or age). Modeling such an XR environment at various timescales can be computationally intensive, particularly when modeling the XR environment over larger timescales. Accordingly, in various implementations, different models are used to determine the environment state of the XR environment when presenting the XR environment at different timescales.

First claim

What is claimed is:

1. A method comprising:
at an electronic device including a processor and non-transitory memory:
obtaining a first environment state associated with a first environment time of an environment and a first timestep value;
in accordance with a determination that the first timestep value satisfies a first criterion, selecting a first model from among a plurality of models and determining, according to the first model and based on the first environment state, a second environment state associated with a second environment time; and
in accordance with a determination that the first timestep value satisfies a second criterion different from the first criterion, selecting a second model different from the first model from among a plurality of models and determining, according to the second model and based on the first environment state, a third environment state associated with a third environment time.

2. The method of claim 1 , wherein the first environment state includes an XML file.

3. The method of claim 1 , wherein the first environment state, second environment state, and third environment state indicate inclusion in the associated environment of one or more assets, and wherein the first environment state, second environment state, and third environment state include data indicating:
the inclusion of a first asset of the one or more assets;
a type of the first asset;
a respective location of the first asset in the associated environment; and
a respective age of the first asset.

4. The method of claim 1 , further comprising:
displaying the environment having the first environment state at a first time;
displaying the environment having the second environment state at a second time a frame time later than the first time; and
displaying the environment having the third environment state at a third time the frame time later than the second time.

5. The method of claim 1 , wherein the second model is more computationally efficient than the first model.

6. The method of claim 1 , wherein the first environment state indicates inclusion in the environment of a first set of one or more assets and one or more states for the first set of one or more assets, and wherein the second environment state indicates inclusion in the environment of a second set of one or more assets and one or more states for the second set of one or more asset.

7. The method of claim 6 , wherein:
determining, in accordance with the first model, the second environment state includes determining a second value of an asset state of the second environment state based on a first value of the asset state of the first environment state; and
determining, in accordance with the second model, the third environment state includes determining a third value of the asset state of the third environment state independent of the second value of the asset state of the second environment state.

8. The method of claim 6 , wherein:
determining, in accordance with the first model, the second environment state includes determining a second value of an asset state of the second environment state based on the first environment state; and
determining, in accordance with the second model, the third environment state includes forgoing determining a third value of the asset state of the third environment state based on the second environment state.

9. The method of claim 6 , wherein:
determining, in accordance with the first model, the second environment state includes determining a second value of an asset state of the second environment state based on a first value of the asset state of the first environment state; and
determining, in accordance with the second model, the third environment state includes excluding the asset having the asset state from the third environment state.

10. The method of claim 9 , wherein determining, in accordance with the second model, the third environment state includes inclusion of one or more new assets having the same asset type as the asset having the asset state based on a number of assets having a respective asset type in the second environment state.

11. A device comprising:
one or more processors;
a non-transitory memory; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
obtain a first environment state associated with a first environment time of an environment and a first timestep value;
in accordance with a determination that the first timestep value satisfies a first criterion, select a first model from among a plurality of models and determine, according to the first model and based on the first environment state, a second environment state associated with a second environment time; and
in accordance with a determination that the first timestep value satisfies a second criterion different from the first criterion, select a second model different from the first model from among a plurality of models and determine, according to the second model and based on the first environment state, a third environment state associated with a third environment time.

12. The device of claim 11 , wherein the first environment state, second environment state, and third environment state indicate inclusion in the associated environment of one or more assets, and wherein the first environment state, second environment state, and third environment state include data indicating:
the inclusion of a first asset of the one or more assets;
a type of the first asset;
a respective location of the first asset in the associated environment; and
a respective age of the first asset.

13. The device of claim 12 , wherein the one or more programs further cause the device to:
display the environment having the first environment state at a first time;
display the environment having the second environment state at a second time a frame time later than the first time; and
display the environment having the third environment state at a third time the frame time later than the first time.

14. The device of claim 12 , wherein the second model is more computationally efficient than the first model.

15. The device of claim 12 , wherein the first environment state indicates inclusion in the environment of a first set of one or more assets and one or more states for the first set of one or more assets, and wherein the second environment state indicates inclusion in the environment of a second set of one or more assets and one or more states for the second set of one or more asset.

16. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device, cause the device to:
obtain a first environment state associated with a first environment time of an environment and a first timestep value;
in accordance with a determination that the first timestep value satisfies a first criterion, select a first model from among a plurality of models and determine, according to the first model and based on the first environment state, a second environment state associated with a second environment time; and
in accordance with a determination that the first timestep value satisfies a second criterion different from the first criterion, select a second model different from the first model from among a plurality of models and determine, according to the second model and based on the first environment state, a third environment state associated with a third environment time.

17. The non-transitory memory of claim 16 , wherein the first environment state, second environm

Assignees

• Apple Inc

Inventors

• MORGAN Bo
• Drummond Mark E
• Meier Peter
• Dunn Cameron J
• Russell John Christopher
• Sivapurapu Siva Chandra Mouli
• Richter Ian M

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F40/14

  Tree-structured documents (parsing G06F40/205; validation G06F40/226) · CPC title

• A63F13/55

  Controlling game characters or game objects based on the game progress · CPC title

• A63F2300/60

  Methods for processing data by generating or executing the game program · CPC title