Language model adaptation

Patent metadata

Field	Value
Publication number	US-12014726-B2
Application number	US-202217706057-A
Country	US
Kind code	B2
Filing date	Mar 28, 2022
Priority date	May 30, 2019
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

Exemplary embodiments relate to adapting a generic language model during runtime using domain-specific language model data. The system performs an audio frame-level analysis, to determine if the utterance corresponds to a particular domain and whether the ASR hypothesis needs to be rescored. The system processes, using a trained classifier, the ASR hypothesis (a partial hypothesis) generated for the audio data processed so far. The system determines whether to rescore the hypothesis after every few audio frames (representing a word in the utterance) are processed by the speech recognition system.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving audio data representing an utterance, the audio data comprising:
a first portion representing a first portion of the utterance, and
a second portion representing a second portion of the utterance subsequent to the first portion of the utterance;

performing automatic speech recognition (ASR) processing by an ASR processing component and using the first portion of the audio data to determine first ASR results data;
processing the first ASR results data, using a trained model, to determine that the utterance corresponds to a first domain;
determining first data representing the first domain; and
performing ASR processing using the first data and the second portion of the audio data to determine second ASR results data.

2. The computer-implemented method of claim 1 , wherein processing the first ASR results data to determine that the utterance corresponds to the first domain comprises:
determining the first ASR results data represents a first word; and
determining the first word corresponds to the first domain.

3. The computer-implemented method of claim 1 , wherein determining the first data comprises determining ASR language model data.

4. The computer-implemented method of claim 3 , wherein the first data corresponds to ASR language model weight data and wherein performing ASR processing using the first data and the second portion of the audio data comprises:
generating an updated ASR language model using the ASR language model weight data; and
using the updated ASR language model to process acoustic model results data representing the second portion of the utterance.

5. The computer-implemented method of claim 1 , wherein performing ASR processing using the first portion of the audio data to determine the first ASR results data comprises:
processing the first portion of the audio data using an acoustic model to determine acoustic model results data; and
processing the acoustic model results data using an ASR language model to determine the first ASR results data.

6. The computer-implemented method of claim 1 , wherein determining the first data comprises determining first finite state transducer (FST) data.

7. The computer-implemented method of claim 6 , further comprising:
processing the first FST data and second FST data corresponding to a generic ASR language model to determine third FST data corresponding to an updated ASR language model,
wherein determining the second ASR results data comprises using the updated ASR language model to process acoustic model results data representing the second portion of the utterance.

8. The computer-implemented method of claim 1 , further comprising:
determining at least one adjusted ASR weight value corresponding to the first domain,
wherein performing ASR processing using the first data comprises using the at least one adjusted ASR weight value.

9. The computer-implemented method of claim 8 , wherein the at least one adjusted ASR weight value corresponds to language model N-gram.

10. A system comprising:
at least one processor; and
at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
receive audio data representing an utterance, the audio data comprising:
a first portion representing a first portion of the utterance, and
a second portion representing a second portion of the utterance subsequent to the first portion of the utterance;

perform automatic speech recognition (ASR) processing by at least one ASR component and using the first portion of the audio data to determine first ASR results data;
process the first ASR results data, using a trained model, data to determine that the utterance corresponds to a first domain;
determine first data representing the first domain; and
perform ASR processing using the first data and the second portion of the audio data to determine second ASR results data.

11. The system of claim 10 , wherein the instructions that cause the system to process the first ASR results data to determine that the utterance corresponds to the first domain comprise instructions that, when executed by the at least one processor, cause the system to:
determine the first ASR results data represents a first word; and
determine the first word corresponds to the first domain.

12. The system of claim 10 , wherein the first data corresponds to ASR language model data.

13. The system of claim 12 , wherein the first data corresponds to ASR language model weight data and wherein the instructions that cause the system to perform ASR processing using the first data and the second portion of the audio data comprise instructions that, when executed by the at least one processor, cause the system to:
generate an updated ASR language model using the ASR language model weight data; and
use the updated ASR language model to process acoustic model results data representing the second portion of the utterance.

14. The system of claim 10 , wherein the instructions that cause the system to perform ASR processing using the first portion of the audio data to determine first ASR results data comprise instructions that, when executed by the at least one processor, cause the system to:
process the first portion of the audio data using an acoustic model to determine acoustic model results data; and
process the acoustic model results data using an ASR language model to determine the first ASR results data.

15. The system of claim 10 , wherein the first data includes first finite state transducer (FST) data.

16. The system of claim 15 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
process the first FST data and second FST data corresponding to a generic ASR language model to determine third FST data corresponding to an updated ASR language model,
wherein determination of the second ASR results data comprises using the updated ASR language model to process acoustic model results data representing the second portion of the utterance.

17. The system of claim 10 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
determine at least one adjusted ASR weight value corresponding to the first domain,
wherein performance of ASR processing using the first data comprises using the at least one adjusted ASR weight value.

18. The system of claim 17 , wherein the at least one adjusted ASR weight value corresponds to language model N-gram.

Assignees

• Amazon Tech Inc

Inventors

• Gandhe Ankur
• Rastrow Ariya
• Maas Roland Maximilian Rolf
• Hoffmeister Bjorn

Classifications

• G10L15/063

  Training · CPC title

• G10L15/01

  Assessment or evaluation of speech recognition systems · CPC title

• G10L15/083

  Recognition networks (G10L15/142, G10L15/16 take precedence) · CPC title

• G10L15/197

  Probabilistic grammars, e.g. word n-grams · CPC title

• G10L15/065 Primary

  Adaptation · CPC title