Systems and methods for speech recognition in unseen and noisy channel conditions

Patent metadata

Field	Value
Publication number	US-11217228-B2
Application number	US-201716085262-A
Country	US
Kind code	B2
Filing date	Mar 22, 2017
Priority date	Mar 22, 2016
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

Systems and methods for speech recognition are provided. In some aspects, the method comprises receiving, using an input, an audio signal. The method further comprises splitting the audio signal into auditory test segments. The method further comprises extracting, from each of the auditory test segments, a set of acoustic features. The method further comprises applying the set of acoustic features to a deep neural network to produce a hypothesis for the corresponding auditory test segment. The method further comprises selectively performing one or more of: indirect adaptation of the deep neural network and direct adaptation of the deep neural network.

First claim

The invention claimed is:

1. A method for speech recognition comprising:
receiving, using an input, an audio signal;
splitting the audio signal into auditory test segments;
extracting, from each of the auditory test segments, a set of acoustic features;
applying the set of acoustic features to a deep neural network to produce a hypothesis for the corresponding auditory test segment; and
selectively performing one or more of: indirect adaptation of the deep neural network and direct adaptation of the deep neural network.

2. The method of claim 1 , wherein performing indirect adaptation of the deep neural network comprises:
extracting, from each of the auditory test segments, two distinct sets of acoustic features; and
applying the two distinct sets of acoustic features to the deep neural network simultaneously.

3. The method of claim 2 , further comprising:
performing a feature-space transformation on each of the two distinct sets of acoustic features prior to applying the two distinct sets of acoustic features to the deep neural network simultaneously.

4. The method of claim 3 , wherein the feature-space transformation is a feature space maximum likelihood linear regression transformation.

5. The method of claim 1 , wherein the set of acoustic features comprises a set of feature vectors, each of the set of feature vectors comprising quantitative measures of acoustic properties of the corresponding auditory test segment.

6. The method of claim 5 , wherein the quantitative measures of acoustic properties comprise at least one of gammatone filterbank energies, normalized modulation coefficients, mel-filterbank energies, and mel-frequency cepstral coefficients.

7. The method of claim 1 , wherein the deep neural network is pre-trained using transcribed audio signals.

8. The method of claim 7 , further comprising:
applying the set of acoustic features to a deep autoencoder to produce (i) a set of deep autoencoder bottleneck features, and (ii) a set of recovered acoustic features based on an inverse operation by the deep autoencoder on the set of deep autoencoder bottleneck features.

9. The method of claim 8 , wherein the set of deep autoencoder bottleneck features is used to extract an entropy-based confidence measure for the corresponding auditory test segment.

10. The method of claim 9 , wherein performing direct adaptation of the deep neural network comprises:
selecting, using the entropy-based confidence measure, the auditory test segments having percentile cumulative entropies below a threshold percentile cumulative entropy; and
retraining the deep neural network using the selected auditory test segments.

11. The method of claim 8 , wherein the deep autoencoder is pre-trained with transcribed audio signals using mean squared error backpropagation.

12. The method of claim 1 , wherein the deep neural network is one of a convolutional neural network, a time-convolutional neural network, and a time-frequency convolutional neural network.

13. A speech recognition system comprising:
an input configured to receive an audio signal;

a processor and a memory having instructions executable by the processor, causing the processor to:
receive, using the input, the audio signal;
split the audio signal into auditory test segments;
extract, from each of the auditory test segments, a set of acoustic features;
apply the set of acoustic features to a deep neural network to produce a hypothesis for the corresponding auditory test segment;
selectively perform one or more of: indirect adaptation of the deep neural network and direct adaptation of the deep neural network; and
an output configured to transmit the hypothesis.

14. The speech recognition system of claim 13 , wherein the deep neural network is pre-trained using transcribed audio signals.

15. The speech recognition system of claim 14 , wherein, when performing indirect adaptation of the deep neural network, the processor is configured to:
extract, from each of the auditory test segments, two distinct sets of acoustic features; and
apply the two distinct sets of acoustic features to the deep neural network simultaneously.

16. The speech recognition system of claim 15 , wherein the processor is further configured to:
perform a feature-space transformation on each of the two distinct sets of acoustic features prior to applying the two distinct sets of acoustic features to the deep neural network simultaneously.

17. The speech recognition system of claim 16 , wherein the feature-space transformation is a feature space maximum likelihood linear regression transformation.

18. The speech recognition system of claim 14 , wherein the processor is further configured to:
apply the set of acoustic features to a deep autoencoder to produce (i) a set of deep autoencoder bottleneck features, and (ii) a set of recovered acoustic features based on an inverse operation by the deep autoencoder on the set of deep autoencoder bottleneck features.

19. The speech recognition system of claim 18 , wherein the set of deep autoencoder bottleneck features is used by the processor to extract an entropy-based confidence measure for the corresponding auditory test segment.

20. The speech recognition system of claim 19 , wherein performing direct adaptation of the deep neural network comprises:
selecting, using the entropy-based confidence measure, the auditory test segments with percentile cumulative entropies below a threshold percentile cumulative entropy; and
retraining the deep neural network using the selected auditory test segments.

Assignees

• Stanford Res Inst Int

Inventors

• Mitra Vikramjit
• Franco Horacio E
• Bartels Chris D
• Vergyri Dimitra
• Van Hout Julien
• Graciarena Martin

Classifications

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L25/24

  the extracted parameters being the cepstrum · CPC title

• G10L15/02

  Feature extraction for speech recognition; Selection of recognition unit · CPC title

• G10L15/16 Primary

  using artificial neural networks · CPC title

• G10L15/075 Primary

  supervised, i.e. under machine guidance · CPC title