Planner for an objective-effectuator

Patent metadata

Field	Value
Publication number	US-12394157-B2
Application number	US-202217685471-A
Country	US
Kind code	B2
Filing date	Mar 3, 2022
Priority date	May 6, 2019
Publication date	Aug 19, 2025
Grant date	Aug 19, 2025

Abstract

In some implementations, a method includes obtaining an objective for a computer-generated reality (CGR) representation of an objective-effectuator. In some implementations, the objective is associated with a plurality of time frames. In some implementations, the method includes determining a plurality of candidate plans that satisfy the objective. In some implementations, the method includes selecting a first candidate plan of the plurality of candidate plans based on a selection criterion. In some implementations, the method includes effectuating the first candidate plan in order to satisfy the objective. In some implementations, the first candidate plan triggers the CGR representation of the objective-effectuator to perform a series of actions over the plurality of time frames associated with the objective.

First claim

What is claimed is:

1. A method comprising:
at a device including a non-transitory memory, a display, and one or more processors coupled with the non-transitory memory and the display:
displaying, on the display, a computer-generated reality (CGR) environment that is associated with state information that indicates one or more states of the CGR environment, wherein displaying the CGR environment is based on a first portion of the state information that is accessible to an objective-effectuator and a second portion of the state information that is not accessible to the objective-effectuator;
determining, based on the first portion of the state information that is accessible to the objective-effectuator and not the second portion of the state information that is not accessible to the objective-effectuator, a candidate plan that provides guidance for generating a series of actions that satisfies an objective of the objective-effectuator; and
while displaying the CGR environment, displaying, on the display, a CGR representation of the objective-effectuator performing the series of actions that is generated in accordance with the guidance provided by the candidate plan in order to advance the objective of the objective-effectuator.

2. The method of claim 1 , wherein the one or more states include a current state of the CGR environment.

3. The method of claim 1 , wherein the one or more states include a set of one or more historical states of the CGR environment.

4. The method of claim 1 , wherein the CGR environment is associated with a time duration;
wherein the state information indicates an initial state of the CGR environment at a beginning of the time duration; and
wherein the candidate plan is a function of the initial state of the CGR environment at the beginning of the time duration.

5. The method of claim 4 , wherein the initial state of the CGR environment indicates initial positions of CGR objects within the CGR environment at the beginning of the time duration; and
wherein the candidate plan is a function of the initial positions of the CGR objects at the beginning of the time duration.

6. The method of claim 4 , wherein the initial state of the CGR environment indicates initial positions of virtual characters and virtual equipment in the CGR environment at the beginning of the time duration; and
wherein the candidate plan is a function of the initial positions of the virtual characters and the virtual equipment in the CGR environment.

7. The method of claim 1 , wherein the CGR environment is associated with a time duration;
wherein the state information indicates an end state of the CGR environment at an end of the time duration; and
wherein the candidate plan is a function of the end state of the CGR environment at the end of the time duration.

8. The method of claim 7 , wherein the end state of the CGR environment indicates end positions of CGR objects within the CGR environment at the end of the time duration; and
wherein the candidate plan is a function of the end positions of the CGR objects at the end of the time duration.

9. The method of claim 7 , wherein the end state of the CGR environment indicates end positions of virtual characters and virtual equipment in the CGR environment at the end of the time duration; and
wherein the candidate plan is a function of the end positions of the virtual characters and the virtual equipment in the CGR environment.

10. The method of claim 1 , wherein the state information indicates environmental conditions within the CGR environment; and
wherein determining the candidate plan comprises:
forgoing selection of a first candidate plan, from a plurality of candidate plans, that is adversely impacted by the environmental conditions within the CGR environment; and
selecting a second candidate plan, from the plurality of candidate plans, based on the environmental conditions within the CGR environment.

11. The method of claim 1 , wherein determining the candidate plan comprises selecting the candidate plan from a plurality of candidate plans.

12. The method of claim 11 , wherein selecting the candidate plan comprises selecting the candidate plan based on a selection criterion.

13. The method of claim 12 , wherein selecting the candidate plan based on the selection criterion comprises selecting the candidate plan based on respective confidence scores of the plurality of candidate plans.

14. The method of claim 1 , wherein displaying the CGR representation of the objective-effectuator corresponds to adding the CGR representation of the objective-effectuator to the display while displaying the CGR environment.

15. The method of claim 1 , wherein the guidance indicates types of actions that can be generated for the objective-effectuator.

16. The method of claim 1 , wherein the guidance indicates a permissible set of actions and an impermissible set of actions.

17. A device comprising:
one or more processors;
a non-transitory memory;
a display; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
display, on the display, a computer-generated reality (CGR) environment that is associated with state information that indicates one or more states of the CGR environment, wherein displaying the CGR environment is based on a first portion of the state information that is accessible to an objective-effectuator and a second portion of the state information that is not accessible to the objective-effectuator;
determine, based on the first portion of the state information that is accessible to the objective-effectuator and not the second portion of the state information that is not accessible to the objective-effectuator, a candidate plan that provides guidance for generating a series of actions that satisfies an objective of the objective-effectuator; and
while displaying the CGR environment, display, on the display, a CGR representation of the objective-effectuator performing the series of actions that is generated in accordance with the guidance provided by the candidate plan in order to advance the objective of the objective-effectuator.

18. The device of claim 17 , wherein the CGR environment is associated with a time duration;
wherein the state information indicates an initial state of the CGR environment at a beginning of the time duration; and
wherein the candidate plan is a function of the initial state of the CGR environment at the beginning of the time duration.

19. The device of claim 18 , wherein the initial state of the CGR environment indicates initial positions of CGR objects within the CGR environment at the beginning of the time duration; and
wherein the candidate plan is a function of the initial positions of the CGR objects at the beginning of the time duration.

20. The device of claim 18 , wherein the initial state of the CGR environment indicates initial positions of virtual characters and virtual equipment in the CGR environment at the beginning of the time duration; and
wherein the candidate plan is a function of the initial positions of the virtual characters and the virtual equipment in the CGR environment.

21. The device of claim 17 , wherein the state information indicates environmental conditions within the CGR environment; and
wherein determining the candidate plan comprises:
foregoing selection of a first candidate plan, from a plurality of candidate plans, that is adversely impacted by the environmental conditions wi

Assignees

• Apple Inc

Inventors

• Drummond Mark
• Sivapurapu Siva Chandra Mouli
• MORGAN Bo

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06N20/00

  Machine learning · CPC title