System and method for learning alternate pronunciations for speech recognition

Patent metadata

Field	Value
Publication number	US-9767792-B2
Application number	US-201615291353-A
Country	US
Kind code	B2
Filing date	Oct 12, 2016
Priority date	Oct 16, 2013
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A system and method for learning alternate pronunciations for speech recognition is disclosed. Alternative name pronunciations may be covered, through pronunciation learning, that have not been previously covered in a general pronunciation dictionary. In an embodiment, the detection of phone-level and syllable-level mispronunciations in words and sentences may be based on acoustic models trained by Hidden Markov Models. Mispronunciations may be detected by comparing the likelihood of the potential state of the targeting pronunciation unit with a pre-determined threshold through a series of tests. It is also within the scope of an embodiment to detect accents.

First claim

The invention claimed is:

1. A method for generating candidate pronunciations for a selected word for learning alternative pronunciations of the word in a given language utilizing a grammar-based recognizer in a speech recognition system, the method comprising the steps of:
a. training an acoustic model for use by the grammar-based recognizer on a large speech corpus to distinguish phonemes;
b. constructing a phoneme confusion matrix for application by the grammar-based recognizer to find similar phonemes of mispronounced phonemes in the selected word;
c. constructing a phoneme replacement candidate list of the selected word for each phoneme in a set of speech data containing pronunciations for recognition, using the phoneme confusion matrix;
d. learning, by the grammar-based recognizer, candidate pronunciations of the word, using input from the acoustic model;
e. combining said learned candidate pronunciations with a linguistic dictionary to create a pooled dictionary; and
f. pruning said pooled dictionary to limit the number of learned candidate pronunciations in order to create an improved dictionary.

2. The method of claim 1 , wherein the acoustic model in step (a) is trained by one of: maximum likelihood criterion and discriminative training criterion.

3. The acoustic model of claim 1 , wherein said acoustic model in step (a) is based on a Hidden Markov Model and Gaussian Mixture Model.

4. The method of claim 1 , wherein the constructing of step (b) comprises merging an acoustic confusion matrix with a linguistic confusion matrix.

5. The acoustic confusion matrix of claim 4 , wherein a low value in the acoustic confusion matrix indicates a phoneme is similar to other phonemes and confusable.

6. The linguistic confusion matrix of claim 4 , wherein the linguistic confusion matrix comprises a binary matrix comprising the numbers 0 and 1.

7. The linguistic confusion matrix of claim 6 , wherein 0 indicates that a phoneme belongs to the same confusion cluster as an other phoneme and the phonemes are confusable.

8. The method of claim 1 , wherein the constructing of step (c) further comprises the steps of:
a. selecting a phoneme from the speech data as a target phoneme for analysis and arranging the remaining phonemes based on distance to the target phoneme;
b. applying a statistical clustering algorithm to similarly group the arranged phonemes;
c. constructing the list of phoneme replacement candidates for the target phoneme from the similarly grouped phonemes; and
d. repeating all of the steps for each phoneme in the speech data set.

9. The method of claim 8 , wherein the distance between a phoneme and said target phoneme in step (a) represents a confusion value in a phoneme confusion matrix.

10. The phoneme confusion matrix of claim 9 , wherein a low value indicates high confusion between a phoneme and a target phoneme.

11. The method of claim 1 , wherein the learning of step (d) further comprises:
a. obtaining an original pronunciation for the selected word that has been misrecognized;
b. generating an alternative pronunciation data set for the selected word, wherein an improved new pronunciation is compared with an original pronunciation;
c. performing recognition on the alternative pronunciation data set, the acoustic model, and the set of speech data;
d. determining a best pronunciation from the alternative pronunciation data set; and
e. retaining selected pronunciations from said alternative pronunciation data set, wherein the selected pronunciations are retained to form a learned pronunciation data set.

12. The method of claim 11 , wherein the original pronunciation in step (a) is obtained from one of: a linguistic dictionary and an automatic word-to-phoneme generator.

13. The method of claim 11 , wherein generating an alternative pronunciation data set further comprises:
a. placing groups of phonemes in their respective positions; and
b. obtaining all possible phoneme combinations of the phonemes.

14. The method of claim 11 , wherein the determination of the best pronunciation comprises basing the determination on pronunciation which results in the highest recognition accuracy.

15. The method of claim 11 , wherein the recognition of step (c) is performed using a Viterbi decoding algorithm.

16. The method of claim 11 , wherein step (b) further comprises the step of: determining the size of the alternative pronunciation data set by the mathematical equation:
X=Π m=1 M N m .
where M represents a number of phonemes and N m represents a number of phoneme candidates.

17. The linguistic dictionary of claim 1 , wherein the linguistic dictionary comprises a set of pronunciations of common words in a language.

18. The method of claim 1 , wherein creation of the improved dictionary in step (f) further comprises the steps of:
a. determining a distance from each word to an other word in the linguistic dictionary;
b. creating a subset of similar words for each misrecognized word;
c. performing recognition on the subset of similar words, the acoustic model and the set of speech;
d. identifying a frequency of failure;
e. removing a pronunciation contributing to frequency failure greater than a threshold; and
f. repeating the process for all misrecognized words.

19. The method of claim 18 , wherein identifying a frequency of failure further comprises identifying: incorrect recognitions, a pronunciation associated with an incorrect recognition, and the frequency of failure related to an incorrect recognition.

20. The method of claim 1 further comprising the step of optimizing the efficiency of learning candidate pronunciations, wherein said optimizing comprises one or more of the following:
a. reducing the length of a phoneme replacement candidate list for each phoneme in the original pronunciation of a word that has been mispronounced if a number of candidate pronunciations exceed a threshold; and
b. optimizing a phoneme determination order when obtaining a desired pronunciation for a misrecognized word.

21. The method of claim 20 , wherein step (a) further comprises the step of determining the scale of length reduction of a phoneme replacement candidate list with the mathematical equation:

r
m
′

=

(

M
max

-
1

M
-
1

)

⁢

r
m

wherein M represents a length of a phoneme sequence, M max represents a threshold of the phoneme sequence, and r m represents a threshold of a search radius.

22. The method of claim 20 , wherein the phoneme determination order in step (b) continues from a phoneme with the longest phoneme replacement candidate list and continues in the descending order of the length of phoneme replacement candidate list for each phoneme.

23. A method for learning alternative pro

Assignees

• Interactive Intelligence Group Inc

Inventors

• Ge Zhenhao
• Tyagi Vivek
• Ganapathiraju Aravind
• Iyer Ananth Nagaraja
• Randal Scott Allen
• Wyss Felix Immanuel

Classifications

• G09B19/04

  Speaking (with audible presentation of the material to be studied G09B5/04) · CPC title

• G10L15/187

  Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams · CPC title

• G10L15/063 Primary

  Training · CPC title

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G09B19/06

  Foreign languages (with audible presentation of material to be studied G09B5/04) · CPC title