Mediation of wakeword response for multiple devices

Patent metadata

Field	Value
Publication number	US-9996316-B2
Application number	US-201514868170-A
Country	US
Kind code	B2
Filing date	Sep 28, 2015
Priority date	Sep 28, 2015
Publication date	Jun 12, 2018
Grant date	Jun 12, 2018

Abstract

Devices, systems, and methods provide synchronization of multiple voice-controlled devices to establish priority of one of the devices to respond to an acoustic signal, preventing other devices from responding to a single user command. Each device is configured to mediate its response to a wakeword by, after detecting the wakeword in an audio input, generating a data packet and attempting to send it to the other devices over a communication channel that prevents data collision. One device will succeed in sending the packet, while the other devices cannot until they receive the first device's packet, indicating that the first device has priority. Additionally, the devices may process their audio inputs to determine a signal quality, the signal qualities are shared between all devices, and the device with the best signal quality assumes priority.

First claim

What is claimed is:

1. A computing device comprising:
a network interface;
a microphone;
a processor that converts an acoustic signal received by the microphone into a first audio input; and
memory storing program instructions, the computing device executing the program instructions to:
determine a first signal strength value representing a first signal-to-noise ratio of the first audio input;
determine that the first audio input comprises a wakeword;
generate a first data packet comprising the first signal strength value;
send the first data packet to an access point device communicatively coupled to the computing device;
receive a second data packet from the access point device at a first time, the second data packet comprising the first signal strength value;
receive a third data packet from the access point device at a second time, the third data packet comprising a second signal strength value representing a second signal-to-noise ratio of a second audio input, wherein the second audio input is generated from the acoustic signal by a second computing device that is configured to process the second audio input in response to detecting the wakeword included in the second audio input;
determine that the first signal strength value is approximately equal to or greater than the second signal strength value;
determine that the first time is prior to the second time;

determine that the first audio input further comprises a command;
responsive to determining that the first time is prior to the second time, perform an action associated with the command.

2. The computing device of claim 1 , wherein the computing device further executes the program instructions to, after determining that the first audio input comprises the wakeword:
determine that a predetermined time period has elapsed;
wherein the memory stores an expected packet transmission time, and wherein the predetermined time period has a duration that is at least as long as the expected packet transmission time multiplied by a number of remote devices communicatively coupled to the access point device.

3. The computing device of claim 1 , wherein the computing device further executes the program instructions to:
receive a fourth data packet from the access point device, the fourth data packet comprising a third signal strength value representing a third signal-to-noise ratio of a third audio input, wherein the third audio input is generated from the acoustic signal by a third computing device that is configured to process the third audio input in response to detecting the wakeword included in the third audio input; and
before determining that the first audio input comprises the command:
determine that the first signal strength value is less than the third signal strength value;
refrain from executing the program instructions to determine that the first audio input comprises the command; and
refrain from executing the program instructions to perform the action.

4. A computing device, comprising:
a processor that receives a first audio input representing an acoustic signal; and
memory storing program instructions, the processor executing the program instructions to:
determine that the first audio input comprises a wakeword;
send a first data packet comprising a first device identifier to an access point device communicatively coupled to the computing device;
receive, at a first time, a second data packet from the access point device, the second data packet including the first device identifier;
receive, at a second time, a third packet comprising a second device identifier, the third packet representing that a second device had obtained a second audio input representing the acoustic signal and had determined the second audio input comprises the wakeword;
determine that the second data packet includes the first device identifier;
determine that the first time precedes the second time;
determine that the first audio input further comprises a command responsive to determining that the second data packet includes the first device identifier and that the first time precedes the second time; and
process the command.

5. The computing device of claim 4 , wherein the processor executes the program instructions to:
maintain, for a predetermined time period after determining that the first audio input comprises the wakeword, a connection to the access point device via a communication channel; and
receive at least the second data packet and the third data packet over the communication channel during the predetermined time period; and
subsequent to the predetermined time period elapsing, process the command.

6. The computing device of claim 4 , wherein the processor executes the program instructions to:
determine a first quality value of the first audio input;
send the first data packet comprising the first quality value to the access point device;
receive the third packet comprising a second quality value generated from the second audio input representing the acoustic signal;
determine that the first quality value is approximately equal to or greater than the second quality value; and
determine that the first audio input further comprises a command responsive to determining that the first quality value is approximately equal to or greater than the second quality value.

7. The computing device of claim 6 , wherein the first quality value comprises a first signal-to-noise ratio of the first audio input and the second quality value comprises a second signal-to-noise ratio of the second audio input.

8. The computing device of claim 6 , wherein the first quality value comprises a first automatic gain control setting used to generate the first audio input, and the second quality value comprises a second automatic gain control setting used to generate the second audio input.

9. The computing device of claim 6 , wherein the processor further executes the program instructions to:
receive a fourth data packet comprising a third quality value; and
determine that the third quality value is less than the first quality value.

10. The computing device of claim 6 , wherein before the computing device executes the program instructions to process the command, the computing device executes the program instructions to:
determine that the first quality value is greater than a threshold quality value.

11. A method, comprising:
identifying a wakeword in a first audio input of a computing device disposed within an audible range of an acoustic signal, the first audio input representing the acoustic signal as detected by the computing device;
determining a first quality value of the first audio input;
generating a first data packet comprising the first quality value;
sending, at a first time, the first data packet to a first remote device over a first communication channel;
receiving, at a second time, a second data packet from the remote device, the second data packet comprising a second quality value generated from a corresponding remote audio input of one or more remote audio inputs representing the acoustic signal obtained by the first remote device disposed in the audible range;
determining that the first quality value is greater than or equal to second quality value of the one or more remote quality values;
determining that the first time precedes the second time;
determining that the first audio input further comprises a command responsive to determining that the first quality value is greater than or equal to the second quality value and that that the first time precedes the second time; and
processing the command.

12. The method of claim 11 , wherein sending the first quali

Assignees

• Amazon Tech Inc

Inventors

• Jorgovanovic Milos

Classifications

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• H04M2250/74

  with voice recognition means · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L25/60

  for measuring the quality of voice signals · CPC title

• G10L2015/088

  Word spotting · CPC title