Robust speech recognition in the presence of echo and noise using multiple signals for discrimination

Patent metadata

Field	Value
Publication number	US-2016358602-A1
Application number	US-201514835588-A
Country	US
Kind code	A1
Filing date	Aug 25, 2015
Priority date	Jun 5, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

Systems and methods for speech recognition system having a speech processor that is trained to recognize speech by considering (1) a raw microphone signal that includes an echo signal and (2) different types of echo information signals from an echo cancellation system (and optionally different types of ambient noise suppression signals from a noise suppressor). The different types of echo information signals may include those used for echo cancelation and those having echo information. The speech recognition system may convert the raw microphone signal and different types of echo information signals (and optional noise suppression signals) into spectral features in the form of a vector, and a concatenator to combine the feature vectors into a total vector (for a period of time) that is used to train the speech processor, and during use of the speech processor to recognize speech.

First claim

What is claimed is:

1 . A speech recognition system (SRS) comprising:
a first input to receive a raw microphone signal having a user voice signal based on user speech during a period of time and an echo signal based on sound produced by a speaker during the period of time; a second input to receive a plurality of types of echo information signals during the period of time, each type of echo information signal including information derived from the echo signal by an echo cancellation system; and a trained speech recognition processor to recognize speech based on the raw microphone signal and the plurality of types of echo information signals, wherein the processor was trained at least by inputting a plurality of different samples of raw microphone signals and a plurality of different samples of each of the plurality of the types of echo information signals.

2 . The system of claim 1 , wherein the sound produced by the speaker is produced by the speaker in response to the speaker receiving an electronic reference signal; and
wherein the plurality of types of echo information signals include the electronic reference signal, a linear echo estimate signal, a residual echo estimate signal, and an echo canceller output signal.

3 . The system of claim 2 , further comprising the echo cancellation system comprising:
a microphone input to receive the raw microphone signal; a electronic reference signal input to receive the electronic reference signal; an echo processor to use the raw microphone signal and the electronic reference signal to produce the linear echo estimate signal, the residual echo estimate signal, and the echo canceller output signal; and an output to output the linear echo estimate signal, the residual echo estimate signal, and the echo canceller output signal, to the second input.

4 . The system of claim 1 , wherein the speech recognition processor includes a deep neural network that was trained using:
a plurality of different ones of electronic reference signals, a plurality of different ones of linear echo estimate signals, a plurality of different ones of residual echo estimate signals, and a plurality of different ones of echo canceller output signals.

5 . The system of claim 1 , wherein the speech recognition processor includes:
spectral feature transformers to transform the raw microphone signal and different types of echo information signals into spectral features in the form of feature vectors; and a concatenator to combine the feature vectors into a total vector that represents the raw microphone signal and different types of echo information signals for a period of time.

6 . The system of claim 5 , wherein the spectral feature transformers and the concatenator are one of (1) used to train the speech recognition processor based on recorded audio information, or (2) used to recognize user speech while using the system.

7 . The system of claim 1 , wherein the plurality of types of echo information signals include different types of ambient noise suppression signals from a noise suppressor.

8 . A method of speech recognition comprising:
receiving at a first input, a raw microphone signal having a user voice signal based on user speech during a period of time and an echo signal based on sound produced by a speaker during the period of time; receiving at a second input, a plurality of types of echo information signals during the period of time, each type of echo information signal including information derived by an echo cancellation system from the echo signal; and recognizing speech at a trained speech recognition processor, based on the raw microphone signal and the plurality of types of echo information signals, wherein the processor was trained at least by inputting a plurality of different samples of raw microphone signals and a plurality of different samples of each of the plurality of the types of echo information signals.

9 . The method of claim 8 , further comprising:
the speaker receiving an electronic reference signal; and the speaker producing the sound produced by the speaker in response to the speaker receiving an electronic reference signal, wherein the plurality of types of echo information signals include the electronic reference signal, a linear echo estimate signal, a residual echo estimate signal, and an echo canceller output signal, and wherein the plurality of types of echo information signals are derived by the echo cancellation system from the echo signal, the raw microphone signal, and the reference signal.

10 . The method of claim 9 , further comprising:
receiving the raw microphone signal at a microphone input of the echo cancelation system; receiving the electronic reference signal at an electronic reference signal input of the echo cancelation system; an echo processor of the echo cancellation system using the raw microphone signal and the electronic reference signal to produce the linear echo estimate signal, the residual echo estimate signal, and the echo canceller output signal; and an output outputting the linear echo estimate signal, the residual echo estimate signal, and the echo canceller output signal to the second input.

11 . The method of claim 9 , wherein the speech recognition processor includes a deep neural network that was trained using:
a plurality of different ones of electronic reference signals, a plurality of different ones of linear echo estimate signals, a plurality of different ones of residual echo estimate signals, and a plurality of different ones of echo canceller output signals.

12 . The method of claim 8 , wherein recognizing speech at the trained speech recognition processor includes:
spectral feature transformers transforming the raw microphone signal and different types of echo information signals into spectral features in the form of feature vectors; and a concatenator combining the feature vectors into a total vector that represents the raw microphone signal and different types of echo information signals for a period of time.

13 . The method of claim 12 , wherein the spectral feature transformers and the concatenator are used to recognize user speech while using the system.

14 . The method of claim 8 , further comprising:
a noise suppressor producing different types of ambient noise suppression signals from the raw microphone signal, wherein the plurality of types of echo information signals include the different types of ambient noise suppression signals.

15 . A method of training a speech recognition system (SRS) comprising:
receiving at a first input, a plurality of raw microphone signals having a user voice signal based on a sample of user speech during a period of time and an echo signal based on a sample of sound produced by a speaker during the period of time; receiving at a second input, a plurality of a plurality of types of echo information signals during the period of time, each type of the plurality of echo information signal including information derived by an echo cancellation system from the echo signal; and training a speech recognition processor to recognize speech based on the raw microphone signals and the plurality of the plurality of types of echo information signals.

16 . The method of claim 15 , further comprising:
the speaker receiving a plurality of electronic reference signal; and the speaker producing the plurality of raw microphone signals in response to the speaker receiving the electronic reference signals, wherein the plurality of the plurality of types of echo information signals include the plurality

Assignees

• Apple Inc

Inventors

• Krishnaswamy Arvindh
• Clark Charles P
• Malik Sarmad

Classifications

• G10L15/16

  using artificial neural networks · CPC title

• G10L15/20 Primary

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title

• G10L2021/02082

  the noise being echo, reverberation of the speech · CPC title

• G10L21/0208

  Noise filtering · CPC title