Token-level interpolation for class-based language models

Patent metadata

Field	Value
Publication number	US-9734826-B2
Application number	US-201514644976-A
Country	US
Kind code	B2
Filing date	Mar 11, 2015
Priority date	Mar 11, 2015
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

Optimized language models are provided for in-domain applications through an iterative, joint-modeling approach that interpolates a language model (LM) from a number of component LMs according to interpolation weights optimized for a target domain. The component LMs may include class-based LMs, and the interpolation may be context-specific or context-independent. Through iterative processes, the component LMs may be interpolated and used to express training material as alternative representations or parses of tokens. Posterior probabilities may be determined for these parses and used for determining new (or updated) interpolation weights for the LM components, such that a combination or interpolation of component LMs is further optimized for the domain. The component LMs may be merged, according to the optimized weights, into a single, combined LM, for deployment in an application scenario.

First claim

What is claimed is:

1. An automatic speech recognition (ASR) system comprising:
an acoustic sensor configured to convert speech into acoustic information;
an acoustic model (AM) configured to convert the acoustic information into a first corpus of words; and
a language model (LM) configured to convert the first corpus of words into plausible word sequences, the LM determined from an interpolation of a plurality of component LMs and corresponding set of coefficient weights, wherein at least one of the component LMs is class-based, and wherein the interpolation is context-specific;
wherein the coefficient weights are determined at least in part based on a set of alternative parses of a training corpus of words;
wherein the coefficient weights are determined according to a process comprising:
determining a training LM interpolation from the component LMs and the set of coefficient weights,
utilizing the training LM interpolation to determine the set of alternative parses of the training corpus,
determining a posterior probability for each of the parses in the set of alternative parses, thereby forming a set of posterior probabilities; and
based on the set of posterior probabilities, determining updated values of the coefficient weights.

2. The system of claim 1 , wherein the ASR system is deployed on a user device.

3. The system of claim 1 , wherein the LM is determined by merging the interpolated component LMs into a single unified LM according to the corresponding set of coefficient weights.

4. The system of claim 1 , wherein at least one of the component LMs comprises a word-phrase-entity (WPE) model.

5. The system of claim 1 , wherein the coefficient weights are determined and optimized according to a process comprising:
determining initial values for the weights in the set of coefficient weights;
determining optimized values of the weights in the set of coefficient weights; and
providing the optimized values as the coefficient weights in the set of coefficient weights.

6. The system of claim 5 , wherein determining the optimized values of the weights comprises:
(a) receiving training material comprising a second corpus of one or more words;
(b) determining a training LM interpolation from the component LMs and the set of coefficient weights;
(c) utilizing the training LM interpolation to determine a set of alternative parses of the second corpus;
(d) determining statistical data based on the set of alternative parses determined in step (c);
(e) based on the statistical data determined in step (d), determining updated values of the coefficient weights in the set of coefficient weights, thereby optimizing the coefficient weights;
(f) determining whether the optimization of the coefficient weights is satisfactory; and
(g) based on the determination of whether the optimization of the coefficient weights is satisfactory:
(i) if the optimization is determined to be satisfactory, providing the values of the coefficient weights; and
(ii) if the optimization is determined not to be satisfactory, repeating steps (b) through (g).

7. The system of claim 6 , wherein determining statistical data based on the set of alternative parses determined in step (c) and determining updated values of the coefficient weights in step (d) comprises:
determining a posterior probability for each of the parses in the set of alternative parses, thereby forming a set of posterior probabilities; and
based on the set of posterior probabilities, determining the updated values of the coefficient weights.

8. The system of claim 6 , wherein the optimization of the coefficient weights is satisfactory where it has achieved convergence.

9. The system of claim 6 , wherein determining whether the optimization of the coefficient weights is satisfactory comprises determining that perplexity is no longer decreasing with each iteration of steps (b) through (g).

10. A method for automatic speech recognition of a corpus of words, utilizing an optimized language model (LM), performed by one or more computing devices having a processor and a memory, the method comprising:
receiving training material for a target domain, the training material including a first corpus of one or more words;
receiving a plurality of component LMs;
determining initial values of interpolation coefficients for the component LMs thereby forming a set of interpolation weights;
determining a first LM interpolation based on the component LMs and the initial values of the set of interpolation weights;
for a number of iterations, each iteration using an iteration LM interpolation:
(a) utilizing the iteration LM interpolation to determine a set of alternative parses of the first corpus;
(b) determining posterior probabilities from each of the parses;
(c) determining updated coefficient values for the set of interpolation weights thereby forming a set of updated interpolation weights;
(d) determining an updated LM interpolation based on the component LMs and the set of updated interpolation weights determined in step (c); and
(e) determining an evaluation of the set of updated interpolation weights;
wherein the iteration LM interpolation is the first LM interpolation for the first iteration, and wherein the iteration LM interpolation is the updated LM interpolation determined in step (d) for each subsequent iteration; and wherein the number of iterations is determined based on the evaluation of the set of updated interpolation weights;
combining the component LMs into a single unified LM according to the set of updated interpolation weights;
receiving a speech acoustic signal generated by an acoustic sensor;
generating a second corpus of words that corresponds to the speech acoustic signal; and
utilizing the optimized LM for automatic speech recognition of the second corpus of words.

11. The method of claim 10 , wherein at least one of the component LMs comprises a class-based LM.

12. The method of claim 11 , wherein the first LM interpolation and the iteration LM interpolation comprise a context-specific interpolation.

13. The method of claim 10 , further comprising combining the component LMs into a single unified LM according to the set of updated interpolation weights.

14. The method of claim 10 , wherein determining initial values of interpolation coefficients comprises setting the values of the weights according to a uniform distribution.

15. The method of claim 10 , wherein the set of interpolation weights comprises context-specific weights, and wherein initial values of interpolation coefficients are determined seeding using a set of context-independent weights.

16. The method of claim 10 , wherein evaluating the set of updated interpolation weights comprises:
(i) determining that the interpolation weights have converged on the corpus, or
(ii) for each iteration:
determining a perplexity;
comparing the determined perplexity for a current iteration with the determined perplexity from a preceding iteration; and
based on the comparison, determining that the perplexity is not improving.

17. One or more computer-readable storage devices having computer-executable instructions embodied thereon, that, when executed by a computing system having a processor and memory, cause the computing system to perform a method of automatic speech recognition of a first corpus of words utilizing an optimized language model (LM), the method comprising:
accessing a training corpus, the training corpus comprising one or more words;
receiving a plurality of component LMs;
determining initial values for a set

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Levit Michael
• Parthasarathy Sarangarajan
• Stolcke Andreas
• Chang Shuangyu

Classifications

• G10L15/063 Primary

  Training · CPC title

• G10L15/197

  Probabilistic grammars, e.g. word n-grams · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G10L15/183 Primary

  using context dependencies, e.g. language models · CPC title