Visual editor for designing augmented-reality effects that utilize voice recognition

Patent metadata

Field	Value
Publication number	US-12243176-B2
Application number	US-202318455534-A
Country	US
Kind code	B2
Filing date	Aug 24, 2023
Priority date	Dec 30, 2020
Publication date	Mar 4, 2025
Grant date	Mar 4, 2025

Abstract

In one embodiment, a computer-implemented method includes receiving, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect. The computer-implemented method further includes receiving, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect. The computer-implemented method further includes configuring, while the AR design tool and the UI are placed into the configure phase, one or more parameters of the voice-command module. The computer-implemented method further includes generating the AR effect utilizing a particular voice command at runtime based on configured one or more parameters of the voice-command module.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receiving, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect;
configuring, while the AR design tool and the UI are placed into the configure phase, one or more parameters of the voice-command module to generate the AR effect based on one or more voice commands; and
generating the AR effect utilizing a particular voice command at runtime based on configured one or more parameters of the voice-command module.

2. The computer-implemented method of claim 1 , wherein the one or more parameters of the voice-command module comprise:
an intent type;
at least one slot; and
one or more entities associated with the at least one slot.

3. The computer-implemented method of claim 1 , wherein generating the AR effect comprises:
determining that the particular voice command at runtime corresponds to the configured one or more parameters of the voice-command module;
selecting a particular parameter of the configured one or more parameters of the voice-command module as a runtime value; and
accessing an AR asset corresponding to selected runtime value.

4. The computer-implemented method of claim 3 , further comprising:
tracking a location of an object in a scene; and
presenting, for display, the AR asset corresponding to the selected runtime value in accordance with a tracked location of the object in the scene.

5. The computer-implemented method of claim 1 , further comprising:
tracking a location of an object in a scene,
wherein the AR effect is generated further based on a tracked location of the object in the scene.

6. The computer-implemented method of claim 1 , wherein configuring the one or more parameters of the voice-command module comprises:
receiving, through the UI, one or more slots for the voice-command module;
receiving, through the UI, one or more entities for the one or more slots; and
configuring the one or more slots to be associated with the one or more entities.

7. The computer-implemented method of claim 1 , wherein the particular voice command comprises one or more of an utterance or one or more words.

8. One or more computer-readable non-transitory non-volatile storage media embodying software that is operable when executed to:
receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect;
configure, while the AR design tool and the UI are placed into the configure phase, one or more parameters of the voice-command module to generate the AR effect based on one or more voice commands; and
generate the AR effect utilizing a particular voice command at runtime based on configured one or more parameters of the voice-command module.

9. The computer-readable non-transitory non-volatile storage media of claim 8 , wherein the one or more parameters of the voice-command module comprise:
an intent type;
at least one slot; and
one or more entities associated with the at least one slot.

10. The computer-readable non-transitory non-volatile storage media of claim 8 , wherein generating the AR effect comprises:
determining that the particular voice command at runtime corresponds to the configured one or more parameters of the voice-command module;
selecting a particular parameter of the configured one or more parameters of the voice-command module as a runtime value; and
accessing an AR asset corresponding to selected runtime value.

11. The computer-readable non-transitory non-volatile storage media of claim 10 , wherein the software is further operable when executed to:
track a location of an object in a scene; and
present, for display, the AR asset corresponding to the selected runtime value in accordance with a tracked location of the object in the scene.

12. The computer-readable non-transitory non-volatile storage media of claim 8 , wherein the software is further operable when executed to:
track a location of an object in a scene,
wherein the AR effect is generated further based on a tracked location of the object in the scene.

13. The computer-readable non-transitory non-volatile storage media of claim 8 , wherein configuring the one or more parameters of the voice-command module comprises:
receiving, through the UI, one or more slots for the voice-command module;
receiving, through the UI, one or more entities for the one or more slots; and
configuring the one or more slots to be associated with the one or more entities.

14. The computer-readable non-transitory non-volatile storage media of claim 8 , wherein the particular voice command comprises one or more of an utterance or one or more words.

15. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:
receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect;
configure, while the AR design tool and the UI are placed into the configure phase, one or more parameters of the voice-command module to generate the AR effect based on one or more voice commands; and
generate the AR effect utilizing a particular voice command at runtime based on configured one or more parameters of the voice-command module.

16. The system of claim 15 , wherein the one or more parameters of the voice-command module comprise:
an intent type;
at least one slot; and
one or more entities associated with the at least one slot.

17. The system of claim 15 , wherein generating the AR effect comprises:
determining that the particular voice command at runtime corresponds to the configured one or more parameters of the voice-command module;
selecting a particular parameter of the configured one or more parameters of the voice-command module as a runtime value; and
accessing an AR asset corresponding to selected runtime value.

18. The system of claim 17 , wherein the processors are further operable when executing the instructions to:
track a location of an object in a scene; and
present, for display, the AR asset corresponding to the selected runtime value in accordance with a tracked location of the object in the scene.

19. The system of claim 15 , wherein the processors are further operable when executing the instructions to:
track a location of an object in a scene,
wherein the AR effect is generated further based on a tracked location of the object in the s

Assignees

• Meta Platforms Inc

Inventors

• Smet Stef Marc
• Verlinde Hannes Luc Herman
• Slater Michael
• Blackburne Benjamin Patrick
• Hariharan Ram Kumar
• Jia Chunjie
• Nisarat Prakarn

Classifications

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06F8/33

  Intelligent editors · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title