Mitigating effects of electronic audio sources in expression detection

Patent metadata

Field	Value
Publication number	US-9734845-B1
Application number	US-201514752400-A
Country	US
Kind code	B1
Filing date	Jun 26, 2015
Priority date	Jun 26, 2015
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

In a speech-based system, a wake word or other trigger expression is used to preface user speech that is intended as a command. The system receives multiple directional audio signals, each of which emphasizes sound from a different direction. The signals are monitored and analyzed to detect the directions of interfering audio sources such as televisions or other types of electronic audio players. One of the directional signals having the strongest presence of speech is selected to be monitored for the trigger expression. If the directional signal corresponds to the direction of an interfering audio source, a more strict standard is used to detect the trigger expression. In addition, the directional audio signal having the second strongest presence of speech may also be monitored to detect the trigger expression.

First claim

The invention claimed is:

1. A system comprising:
a microphone array configured to produce microphone audio signals;
an audio beamformer configured to process the microphone audio signals to produce directional audio signals, wherein a first directional audio signal of the directional audio signals corresponds to a first direction with respect to the microphone array and wherein a second directional audio signal of the directional audio signals corresponds to a second direction with respect to the microphone array, wherein the first directional audio signal and the second directional audio signal emphasize sound from the first direction and the second direction, respectively;
a speech activity detector configured to analyze one or more frequency characteristics of the first directional audio signal and the second directional audio signal to determine a first level of speech presence and a second level of speech presence occurring in the first direction and the second direction, respectively, over time;
a source detector configured to analyze the first level of speech presence and the second level of speech presence occurring over a past time period to determine that an electronic source of sound is located in the first direction or the second direction; and
an expression detector configured to perform actions comprising:
identifying the first direction where a first occurring level of speech presence is a highest level of speech presence;
determining that the first direction corresponds to a direction in which the electronic source of sound is located;
identifying the second direction where a second occurring level of speech presence is a second highest level of speech presence;
analyzing the first directional audio signal corresponding to the first direction to produce a first score indicating a first likelihood that a trigger expression is represented in the first directional audio signal;
analyzing the second directional audio signal corresponding to the second direction to produce a second score indicating a second likelihood that the trigger expression is represented in the second directional audio signal;
comparing the first score to a first threshold;
comparing the second score to a second threshold, wherein the second threshold is less than the first threshold;
determining that (i) the first score is greater than the first threshold or (ii) the second score is greater than the second threshold;
concluding that the trigger expression has been uttered; and
performing speech recognition on subsequent speech, based at least in part on the trigger expression.

2. The system of claim 1 , wherein the source detector is further configured to determine that the electronic source of sound is located in the first direction by performing actions comprising:
determining a first range of frequencies of the first directional audio signal corresponding to the first direction; and
determining that the first range of frequencies is at least partially outside a second range of frequencies, wherein the second range of frequencies corresponds to frequencies of human speech.

3. The system of claim 1 , wherein the source detector is further configured to determine that the electronic source of sound is located in the first direction by determining that the first level of speech presence corresponding to the first direction has exceeded a level threshold for at least a threshold amount of time during the past time period.

4. The system of claim 1 , wherein the speech activity detector is configured to perform actions comprising:
determining one or more features of the first directional audio signal; and
comparing the one or more features to one or more reference signals that are known to contain human speech.

5. The system of claim 1 , wherein analyzing the first directional audio signal to produce the first score comprises:
creating an acoustic model of an utterance represented by the first directional audio signal; and
comparing the acoustic model to one or more reference acoustic models that correspond to the trigger expression.

6. A processor-implemented method, comprising:
receiving, from one or more microphones, a plurality of audio signals, wherein the plurality of audio signals are processed by an audio beamformer to produce directional audio signals, wherein a first audio signal of the directional audio signals corresponds to a first direction with respect to the one or more microphones and wherein a second audio signal of the directional audio signals corresponds to a second direction with respect to the one or more microphones, wherein the first audio signal and the second audio signal emphasize sound from the first direction and the second direction, respectively;
identifying the first direction as an identified direction in which a non-human sound source is located;
analyzing the first audio signal to identify a representation of speech;
determining that the first audio signal corresponds to the identified direction in which the non-human sound source is located;
selecting a first standard to analyze the first audio signal based at least in part on the first audio signal corresponding to the identified direction of the non-human sound source;
analyzing the first audio signal using the first standard to detect an utterance of a trigger expression;
analyzing the second audio signal using a second standard to detect the utterance of the trigger expression, wherein the first standard includes (i) a first threshold that is greater than a second threshold associated with the second standard or (ii) a first detection algorithm that is different than a second detection algorithm associated with the second standard; and
receiving, from the one or more microphones, a third audio signal including subsequent speech for performing subsequent speech recognition, based at least in part on the utterance of the trigger expression.

7. The processor-implemented method of claim 6 , further comprising:
producing a first score indicating a first likelihood that the trigger expression is represented by the first audio signal;
comparing the first score to the first threshold;
producing a second score indicating a second likelihood that the trigger expression is represented by the second audio signal; and
comparing the second score to the second threshold.

8. The processor-implemented method of claim 7 , wherein producing the first score comprises:
creating an acoustic model of the utterance represented by the first audio signal; and
comparing the acoustic model to one or more reference acoustic models that correspond to the trigger expression.

9. The processor-implemented method of claim 6 , wherein:
analyzing the first audio signal using the first standard comprises performing expression detection in accordance with the first detection algorithm; and
analyzing the second audio signal using the second standard comprises performing expression detection in accordance with the second detection algorithm.

10. The processor-implemented method of claim 9 , wherein the first detection algorithm comprises analyzing the first audio signal to detect at least two utterances of the trigger expression within a particular time period to determine a valid utterance of the trigger expression.

11. The processor-implemented method of claim 9 , wherein the first detection algorithm comprises analyzing the first audio signal to detect the utterance of the trigger expression after a time period during which speech is not detected.

12. The processor-implemented method of claim 6 , wherein identifying the first direction as the identified direction further comprises determining

Assignees

• Amazon Tech Inc

Inventors

• Liu Yue
• Jayakumar Praveen
• Chhetri Amit Singh
• Gopalan Ramya

Classifications

• H04R1/406

  microphones · CPC title

• H04R3/005

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G10L25/78 Primary

  Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) · CPC title

• G10L15/00

  Speech recognition (G10L17/00 takes precedence) · CPC title

• H04R2420/09

  Applications of special connectors, e.g. USB, XLR, in loudspeakers, microphones or headphones · CPC title