Meta-data inputs to front end processing for automatic speech recognition

Patent metadata

Field	Value
Publication number	US-9953638-B2
Application number	US-201214411236-A
Country	US
Kind code	B2
Filing date	Jun 28, 2012
Priority date	Jun 28, 2012
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

A computer-implemented method is described for front end speech processing for automatic speech recognition. A sequence of speech features which characterize an unknown speech input provided on an audio input channel and associated meta-data which characterize the audio input channel are received. The speech features are transformed with a computer process that uses a trained mapping function controlled by the meta-data, and automatic speech recognition is performed of the transformed speech features.

First claim

What is claimed is:

1. A method comprising:
receiving, by a computing device, a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the computing device;
receiving meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, and a type of the application;
transforming the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input; and
performing automatic speech recognition of the transformed speech features.

2. The method of claim 1 , wherein the one or more trained mapping functions include at least one of a multilayered perceptron (MLP) mapping function or a deep belief network (DBN) mapping function.

3. The method of claim 2 , wherein the MLP mapping function comprises a mapping function trained on segmentation for the sequence of speech features and the meta-data.

4. The method of claim 1 , wherein the fMMI mapping function comprises a mapping function trained on MLP output posteriors of the MLP mapping function.

5. The method of claim 1 , wherein the application executing on the computing device comprises a messaging application, and the meta-data characterizes the messaging application.

6. The method of claim 1 , wherein the application executing on the computing device comprises a voice-search application, and the meta-data characterizes the voice-search application.

7. The method of claim 1 , wherein the application executing on the computing device comprises a dictation application, and the meta-data characterizes the dictation application.

8. The method of claim 1 , wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce channel impact of the audio input channel.

9. The method of claim 1 , wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce speaker impact of a speaker of the unknown speech input.

10. The method of claim 1 , wherein transforming the sequence of speech features comprises normalizing the sequence of speech features to reduce environmental impact of a source environment in which the unknown speech input is generated.

11. The method of claim 1 , wherein the meta-data characterizes a source environment in which the unknown speech input is generated.

12. The method of claim 1 , wherein the meta-data characterizes a microphone type of the audio input channel.

13. One or more non-transitory computer-readable media storing executable instructions that, when executed by a processor, cause a system to:
receive a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the system;
receive meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, and a type of the application;
transform the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input, wherein transforming the sequence of speech features comprises reducing a dimensionality of the sequence of speech features; and
perform automatic speech recognition of the transformed speech features.

14. The one or more non-transitory computer-readable media of claim 13 , wherein the one or more trained mapping functions include a cepstral variance normalization (CVN) mapping function.

15. The one or more non-transitory computer-readable media of claim 13 , wherein the one or more trained mapping functions include a linear discriminant analysis (LDA) mapping function.

16. The one or more non-transitory computer-readable media of claim 13 , wherein the one or more trained mapping functions include a feature-space maximum likelihood linear regression (fMLLR) mapping function.

17. A system comprising:
at least one processor; and
one or more non-transitory computer-readable media storing executable instructions that, when executed by the at least one processor, cause the system to:
receive a sequence of speech features that characterize an unknown speech input provided on an audio input channel controlled by an application executing on the system;
receive meta-data that characterizes the audio input channel, an audio codec applied when generating the sequence of speech features, a microphone type of the audio input channel, and a type of the application;
transform the sequence of speech features using one or more trained mapping functions including a feature-space maximum mutual information (fMMI) mapping function, the one or more trained mapping functions controlled by the meta-data that characterizes the audio input channel, the audio codec applied when generating the sequence of speech features, the microphone type of the audio input channel, and the type of the application, the fMMI mapping function using neural network based posterior estimates that use the meta-data as input; and
perform automatic speech recognition of the transformed speech features.

18. The system of claim 17 , wherein the microphone type is a built-in microphone, and the meta-data characterizes the audio input channel as the built-in microphone.

19. The system of claim 17 , wherein the microphone type is an external microphone that is external to the system, and the meta-data characterizes the audio input channel as the external microphone.

20. The system of claim 17 , wherein the meta-data further comprises a number representing a size of a data stream after audio encoding of the unknown speech input.

Assignees

• Willett Daniel
• Loeoef Karl Jonas
• Pan Yue
• Pinto Joel
• Gollan Christian
• Nuance Communications Inc

Inventors

• Willett Daniel
• Lööf Karl Jonas
• Pan Yue
• Pinto Joel
• Gollan Christian

Classifications

• G10L15/20 Primary

  Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) · CPC title

• G10L15/14 Primary

  using statistical models, e.g. Hidden Markov Models [HMMs] (G10L15/18 takes precedence) · CPC title

• G10L25/30

  using neural networks · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title