Method and system for robust pattern matching in continuous speech for spotting a keyword of interest using orthogonal matching pursuit

Patent metadata

Field	Value
Publication number	US-9293130-B2
Application number	US-11412808-A
Country	US
Kind code	B2
Filing date	May 2, 2008
Priority date	May 2, 2008
Publication date	Mar 22, 2016
Grant date	Mar 22, 2016

Abstract

A method for speech recognition, the method includes: extracting time-frequency speech features from a series of reference speech elements in a first series of sampling windows; aligning reference speech elements that are not of equal time span duration; constructing a common subspace for the aligned speech features; determining a first set of coefficient vectors; extracting a time-frequency feature image from a test speech stream spanned by a second sampling window; approximating the extracted image in the common subspace for the aligned extracted time-frequency speech features with a second coefficient vector; computing a similarity measure between the first and the second coefficient vector; determining if the similarity measure is below a predefined threshold; and wherein a match between the reference speech elements and a portion of the test speech stream is made in response to a similarity measure below a predefined threshold. The said reference speech elements correspond to a keyword of interest, wherein Simultaneous Orthogonal Matching Pursuit (SOMP) is used in their alignment.

First claim

What is claimed is:

1. A method for speech recognition in mismatched environments, the method comprising:
extracting, by a computing device, time-frequency speech features from a series of reference speech elements in a first series of sampling windows spanning each occurrence of a keyword of interest, wherein the time-frequency speech features represent each reference speech element as a two-dimensional image in the time-frequency plane, wherein extracting time-frequency speech features from the series of reference speech elements is obtained from a feature domain comprising Perceptual Linear Predictive (PLP) modified power spectrum;
aligning, by the computing device, the extracted time-frequency speech features when the reference speech elements from the series of speech elements are not of equal time span duration;
constructing, by the computing device, a sparse representation model common to the aligned extracted time-frequency speech features, using simultaneous sparse approximation of reference speech signals in a time-frequency domain, wherein the simultaneous sparse approximation determines an approximation of a reference speech signal as a linear combination of reference speech signals drawn from a large, linearly dependent collection of reference speech signals;
determining, by the computing device, a first set of coefficient vectors for the aligned extracted time-frequency speech features;
extracting, by the computing device, a time-frequency feature image from a test speech stream spanned by a second sampling window, wherein the reference speech elements and the test speech stream are obtained under mismatched conditions where the test speech stream contains background noise;
approximating, by the computing device, the extracted time-frequency feature image in the sparse representation model for the aligned extracted time-frequency speech features with a second coefficient vector;
computing, by the computing device, a similarity measure between the first set of coefficient vectors and the second coefficient vector;
determining, by the computing device, if the similarity measure is below a predefined threshold; and
wherein a match between the reference speech elements and a portion of the test speech stream spanned by the second sampling window is made in response to the similarity measure being below the predefined threshold, the match indicating the presence of the keyword of interest in the second sampling window;
wherein Simultaneous Orthogonal Matching Pursuit (SOMP) is used for constructing the sparse representation model for the aligned extracted time-frequency speech features by extracting a subspace of common time-frequency structures from different occurrences of the keyword of interest.

2. The method of claim 1 , wherein the method further comprises:
incrementing the second sampling window by a unit of time and extracting a new time-frequency feature image from the test speech stream spanned by the incremented second sampling window;
approximating the extracted new time-frequency feature image in the sparse representation model for the aligned extracted time-frequency speech features with a new coefficient vector;
computing a similarity measure between the first set of coefficient vectors and the new coefficient vector;
determining if the similarity measure is below the predefined threshold;
wherein a match between the reference speech elements and a portion of the test speech stream spanned by the incremented second sampling window is made in response to a similarity measure below the predefined threshold; and
wherein the process of incrementing the second sampling window and computing additional similarity measures continues until the test speech stream is exhausted.

3. The method of claim 1 , wherein the aligning the extracted time-frequency speech features employs Dynamic Time Warping (DTW).

4. The method of claim 1 , wherein the first set of coefficient vectors are obtained by solving least squares equations.

5. A computer-readable storage device encoded with computer instructions that, when executed by a computing device, perform a method for speech recognition in mismatched environments, the method comprising:
extracting time-frequency speech features from a series of reference speech elements in a first series of sampling windows spanning each occurrence of a keyword of interest, wherein the time-frequency speech features represent each reference speech element as a two-dimensional image in the time-frequency plane, wherein extracting time-frequency speech features from the series of reference speech elements is obtained from a feature domain comprising Perceptual Linear Predictive (PLP) modified power spectrum;
aligning the extracted time-frequency speech features when the reference speech elements from the series of speech elements are not of equal time duration;
constructing a sparse representation model common to the aligned extracted time-frequency speech features, using simultaneous sparse approximation of reference speech signals in a time-frequency domain, wherein the simultaneous sparse approximation determines an approximation of a reference speech signal as a linear combination of reference speech signals drawn from a large, linearly dependent collection of reference speech signals;
determining a first set of coefficient vectors for the aligned extracted time-frequency speech features;
extracting a time-frequency feature image from a test speech stream spanned by a second sampling window, wherein the reference speech elements and the test speech stream are obtained under mismatched conditions where the test speech stream contains background noise;
approximating the extracted time-frequency feature image in the sparse representation model for the aligned extracted time-frequency speech features with a second coefficient vector;
computing a similarity measure between the first set of coefficient vectors and the second coefficient vector; and
determining if the similarity measure is below a predefined threshold, wherein a match between the reference speech elements and a portion of the test speech stream spanned by the second sampling window is made in response to the similarity measure being below the predefined threshold, the match indicating the presence of the keyword of interest in the second sampling window;
wherein Simultaneous Orthogonal Matching Pursuit (SOMP) is used for constructing the sparse representation model for the aligned extracted time-frequency speech features by extracting a subspace of common time-frequency structures from different occurrences of the keyword of interest.

6. A computer-readable storage device as defined in claim 5 , wherein the method further comprises:
incrementing the second sampling window by a unit of time and extracting a new time-frequency feature image from the test speech stream spanned by the incremented second sampling window;
approximating the extracted new time-frequency feature image in the sparse representation model for the aligned extracted time-frequency speech features with a new coefficient vector;
computing a similarity measure between the first set of coefficient vectors and the new coefficient vector;
determining if the similarity measure is below the predefined threshold, wherein a match between the reference speech elements and a portion of the test speech stream spanned by the incremented second sampling window is identified in response to a similarity measure below the predefined threshold; and
continuing the process of incrementing the second sampling window and computing additional similarity measures until the test speech stream is exhausted.

7. A computer-readable storage device as defined in claim 5 , wherein aligning the extracted time-frequ

Assignees

• Amini Lisa
• Frossard Pascal
• Kokiopoulou Effrosyni
• Verscheure Oliver
• Nuance Communications Inc

Inventors

• Amini Lisa
• Frossard Pascal
• Kokiopoulou Effrosyni
• Verscheure Oliver

Classifications

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/02 Primary

  Feature extraction for speech recognition; Selection of recognition unit · CPC title