Visual editor for designing augmented-reality effects that utilize voice recognition

Patent metadata

Field	Value
Publication number	US-11790611-B2
Application number	US-202017138096-A
Country	US
Kind code	B2
Filing date	Dec 30, 2020
Priority date	Dec 30, 2020
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

A computer-implemented method, comprising, by an artificial-reality (AR) design tool: receiving, through a user interface (UI) of the AR design tool, instructions to add a voice-command module to an AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities; establishing, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and generate, for the AR effect, an executable program configured to: determine that a detected utterance corresponds to the intent type and includes one or more words associated with the slot; select, based on the one or more words, one of the one or more entities as the runtime value for the slot; send the runtime value to the logic module according to the logical connection.

First claim

What is claimed is:

1. A computer-implemented method, comprising, by an artificial-reality (AR) design tool:
receiving, through a user interface (UI) of the AR design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receiving, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities;
establishing, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and
generating, for the AR effect, an executable program configured to:
detect an utterance at runtime;
determine that the detected utterance at runtime corresponds to the intent type and includes one or more words associated with the slot;
select, based on the one or more words, one of the one or more entities as the runtime value for the slot;
send the runtime value to the logic module according to the logical connection; and
after detecting the utterance at runtime, generate the AR effect based on the runtime value, received over the logical connection, that is selected based on the one or more words that are included by the detected utterance at runtime.

2. The computer-implemented method of claim 1 , further comprising:
customizing the voice-command module, wherein the customizing the voice-command module comprises:
receiving, through the UI, one or more additional entities for the slot; and
configuring the slot to be associated with the one or more additional entities.

3. The computer-implemented method of claim 2 , wherein selecting the runtime value for the slot further comprises:
selecting, based on the one or more words, one of the one or more additional entities as the runtime value for the slot.

4. The computer-implemented method of claim 1 , further comprising:
customizing the voice-command module, including:
receiving, through the UI, an additional slot for the voice-command module;
configuring the additional slot to be associated with one or more additional entities; and
establishing, according to instructions received through the UI, an additional logical connection between the additional slot and the logic module.

5. The computer-implemented method of claim 4 , wherein generating the executable program further comprises:
determining that the detected utterance at runtime corresponds to the intent type and includes one or more particular words associated with the additional slot;
selecting, based on the one or more particular words, one of the one or more additional entities as a particular runtime value for the additional slot; and
sending the particular runtime value to the logic module according to the logical connection.

6. The computer-implemented method of claim 1 , further comprising:
receiving, through the UI, a selection indicating synonyms of the one or more entities of the slot; and
configuring the slot to be associated with the one or more entities and the synonyms of the one or more entities.

7. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effect, the voice-command module having an intent type and at least one slot, the slot associated with one or more entities;
establish, according to instructions received through the UI, a logical connection between the slot and a logic module configured to generate the AR effect depending on a runtime value associated with the slot; and
generate, for the AR effect, an executable program configured to:
detect an utterance at runtime;
determine that the detected utterance at runtime corresponds to the intent type and includes one or more words associated with the slot;
select, based on the one or more words, one of the one or more entities as the runtime value for the slot;
send the runtime value to the logic module according to the logical connection; and
after detecting the utterance at runtime, generate the AR effect based on the runtime value, received over the logical connection, that is selected based on the one or more words that are included by the detected utterance at runtime.

8. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable whene executed to:
customize the voice-command module, wherein customizing the voice-command module comprises:
receiving, through the UI, one or more additional entities for the slot; and
configuring the slot to be associated with the one or more additional entities.

9. The computer-readable non-transitory storage media of claim 8 , wherein selecting the runtime value for the slot further comprises:
selecting, based on the one or more words, one of the one or more additional entities as the runtime value for the slot.

10. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable when executed to:
customize the voice-command module, wherein customizing the voice-command module comprises:
receiving, through the UI, an additional slot for the voice-command module;
configuring the additional slot to be associated with one or more additional entities; and
establishing, according to instructions received through the UI, an additional logical connection between the additional slot and the logic module.

11. The computer-readable non-transitory storage media of claim 10 , wherein generating the executable program further comprises:
determining that the detected utterance at runtime corresponds to the intent type and includes one or more particular words associated with the additional slot;
selecting, based on the one or more particular words, one of the one or more additional entities as a particular runtime value for the additional slot; and
sending the particular runtime value to the logic module according to the logical connection.

12. The computer-readable non-transitory storage media of claim 7 , wherein the software is further operable when executed to:
receive, through the UI, a selection indicating synonyms of the one or more entities of the slot; and
configure the slot to be associated with the one or more entities and the synonyms of the one or more entities.

13. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:
receive, through a user interface (UI) of an artificial-reality (AR) design tool, a selection of a configurable interface element to place the AR design tool and the UI into a configure phase to configure an AR effect;
receive, through the UI of the AR design tool after the AR design tool and the UI are placed into the configure phase in response to selecting the configurable interface element, instructions to add a voice-command module to the AR effe

Assignees

• Meta Platforms Inc

Inventors

• Smet Stef Marc
• Verlinde Hannes Luc Herman
• Slater Michael
• Blackburne Benjamin Patrick
• Hariharan Ram Kumar
• Jia Chunjie
• Nisarat Prakarn

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title