Multi-assistant device control

Patent metadata

Field	Value
Publication number	US-12175976-B2
Application number	US-202217944600-A
Country	US
Kind code	B2
Filing date	Sep 14, 2022
Priority date	Aug 24, 2022
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

A speech-processing system may provide access to one or more virtual assistants via an audio-controlled device. A multi-agent component of a device may be used to coordinate control of device process(es) regardless of the invoked agent and without necessarily sharing information between agents and corresponding speech-processing system(s). Device-state information may be sent from the multi-agent component to a speech-processing system to interpret a user command. If the command relates to a device process control it may be routed to a device skill and to the multi-agent component of the device. The multi-agent component may track a device state and may inform a particular speech-processing system if a process related to the particular speech-processing system has been controlled.

First claim

What is claimed is:

1. A computer-implemented method comprising:
capturing a first utterance by a first device configured to operate with a plurality of speech-processing systems including a first speech-processing system and a second speech-processing system;
processing first audio data representing the first utterance to determine the first utterance includes a first wakeword invoking the first speech-processing system;
sending, from a first component of the first device to the first speech-processing system, the first audio data, the first component configured to operate with regard to the first speech-processing system;
receiving, by the first component from the first speech-processing system, first output data responsive to the first audio data;
sending the first output data from the first component to a second component of the first device, the second component configured to operate with regard to the plurality of speech-processing systems;
processing, by the second component, the first output data to determine a first command to terminate a process involving the first device;
processing, by the second component, state data corresponding to the first device to determine that the process was initiated in response to a command invoking the second speech-processing system;
sending, from the second component to a third component of the first device, an indication to terminate the process, the third component configured to operate with regard to the second speech-processing system; and
in response to the indication, terminating the process.

2. The computer-implemented method of claim 1 , further comprising, prior to capturing the first utterance:
capturing a second utterance by the first device;
processing second audio data representing the second utterance to determine the second utterance includes a second wakeword invoking the second speech-processing system;
sending, from the third component to the second speech-processing system, the second audio data;
receiving, from the second speech-processing system, second output data indicating a second command to start the process;
starting the process using the first device; and
by the second component, determining the state data, wherein the state data indicates the process was initiated in response to a command invoking the second speech-processing system.

3. The computer-implemented method of claim 2 , further comprising:
prior to receiving the first output data, sending the state data from the second component to the first speech-processing system.

4. The computer-implemented method of claim 1 , further comprising:
determining, by the second component, that the process is active; and
after discontinuing the process, sending, from the third component to the second speech-processing system, a second indication that the process was discontinued by the first device.

5. A computer-implemented method comprising:
capturing a first utterance by a first device configured to operate with a plurality of speech-processing systems including a first speech-processing system and a second speech-processing system;
determining the first utterance corresponds to an invocation of the first speech-processing system;
sending, from the first device to the first speech-processing system, first audio data representing the first utterance;
sending, from a component of the first device to the first speech-processing system, first state data representing an active first device process of the first device, the component configured to operate with regard to the plurality of speech-processing systems;
receiving, by the component of the first device, a first command to control the first device process using the first device;
in response to the first command, executing a first operation to control the first device process; and
configuring updated state data corresponding to the first device, the updated state data reflecting execution of the first operation.

6. The computer-implemented method of claim 5 , further comprising, prior to capturing the first utterance:
capturing a second utterance by the first device;
determining the second utterance corresponds to an invocation of the second speech-processing system;
sending, from the first device to the second speech-processing system, second audio data representing the second utterance;
receiving, by the component of the first device, a second command to initiate the first device process using the first device; and
in response to the second command, executing a second operation to initiate the first device process.

7. The computer-implemented method of claim 6 , further comprising, after receiving the first command:
sending, from the first device to the second speech-processing system, an indication corresponding to control of the first device process.

8. The computer-implemented method of claim 6 , further comprising:
prior to receiving the second command:
determining a first user corresponding to the second utterance,
determining the first user corresponds to a first profile, and
sending an indication of the first profile to the second speech-processing system; and

after receiving the second command and prior to receiving the first command:
determining a second user corresponding to the first utterance,
determining the second user corresponds to the first profile, and
sending an indication of the first profile to the first speech-processing system.

9. The computer-implemented method of claim 6 , wherein:
sending the second audio data to the second speech-processing system uses a second component of the first device corresponding to the second speech-processing system; and
sending the first audio data to the first speech-processing system uses a third component of the first device corresponding to the first speech-processing system.

10. The computer-implemented method of claim 5 , further comprising:
sending, from the first device to the first speech-processing system, second data corresponding to at least one active hardware component of the first device.

11. The computer-implemented method of claim 5 , wherein the first state data further corresponds to a second device process controllable by the first device and the first state data indicates the first device process is active and the second device process is inactive.

12. The computer-implemented method of claim 11 , further comprising:
determining, by the component, first priority data corresponding to the first device process;
determining, by the component, second priority data corresponding to the second device process; and
prior to receiving the first command, sending, to the first speech-processing system, the first priority data and the second priority data.

13. A device, comprising:
at least one microphone;
a first component configured to operate with regard to a plurality of speech-processing systems including a first speech-processing system and a second speech-processing system;
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the device to:
capture a first utterance using the at least one microphone;
determine the first utterance corresponds to an invocation of the first speech-processing system;
send, to the first speech-processing system, first audio data representing the first utterance;
send, from the first component to the first speech-processing system, first state data representing an active first device process;
receive, by the first component, a first command to control the first device process;
in response to the first command, execute a first opera

Assignees

• Amazon Tech Inc

Inventors

• Henry David
• Sherritt John Ryan
• Lantin Philippe Andre

Classifications

• G10L15/08

  Speech classification or search · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title