Differential acoustic model representation and linear transform-based adaptation for efficient user profile update techniques in automatic speech recognition

Patent metadata

Field	Value
Publication number	US-9406299-B2
Application number	US-201214399867-A
Country	US
Kind code	B2
Filing date	Mar 8, 2012
Priority date	May 8, 2012
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

A computer-implemented method is described for speaker adaptation in automatic speech recognition. Speech recognition data from a particular speaker is used for adaptation of an initial speech recognition acoustic model to produce a speaker adapted acoustic model. A speaker dependent differential acoustic model is determined that represents differences between the initial speech recognition acoustic model and the speaker adapted acoustic model. In addition, an approach is also disclosed to estimate speaker-specific feature or model transforms over multiple sessions. This is achieved by updating the previously estimated transform using only adaptation statistics of the current session.

First claim

What is claimed is:

1. A method comprising:
adapting, by a computing device, an initial speech recognition acoustic model to produce a speaker adapted acoustic model using speech recognition data from a particular speaker;
developing, by the computing device, a speaker differential acoustic model representing one or more differences between the initial speech recognition acoustic model and the speaker adapted acoustic model;
optimizing different quantization ranges for parameter subsets of the speaker adapted acoustic model using a scoring function;
using the different quantization ranges to minimize an acoustic model difference measure of the speaker adapted acoustic model against the initial speech recognition acoustic model; and
storing the speaker differential acoustic model for subsequent speech recognition with the particular speaker.

2. The method of claim 1 , comprising:
retrieving the stored speaker differential acoustic model; and
processing the speaker differential acoustic model and the initial speech recognition acoustic model to derive the speaker adapted acoustic model.

3. The method of claim 2 , comprising:
using the derived speaker adapted acoustic model to perform speech recognition for the particular speaker.

4. The method of claim 1 , comprising:
ranking the one or more differences based on stored sizes of the one or more differences.

5. The method of claim 1 , comprising:
ranking the one or more differences based on most important differences of the one or more differences.

6. The method of claim 1 , comprising:
ranking the one or more differences using the scoring function;
selecting a subset of the one or more differences, the subset selected based on the ranking the one or more differences;
determining storage-efficient Gaussian component clustering of the speaker differential acoustic model using the scoring function; and
using the subset of the one or more differences and the storage-efficient Gaussian component clustering to minimize the acoustic model difference measure of the speaker adapted acoustic model against the initial speech recognition acoustic model.

7. The method of claim 3 , wherein using the derived speaker adapted acoustic model to perform the speech recognition comprises:
producing a representative recognition output of the speech recognition data from the particular speaker, wherein the representative recognition output is constrained by a recognition language model and search algorithm; and
comparing the representative recognition output to the derived speaker adapted acoustic model.

8. The method of claim 1 , comprising:
determining a session update transform for a linear transform-based speaker adaptation of the speaker differential acoustic model based on speech recognition data from speech utterances by the particular speaker, wherein the linear transform-based speaker adaptation uses relevance smoothing of the speech recognition data; and
producing an updated speaker differential acoustic model by combining the speaker differential acoustic model and the session update transform.

9. A method comprising:
loading a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;
performing speech recognition, by a computing device, for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;
determining a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;
producing an updated user feature transform by combining the initial user feature transform and the session update transform; and
storing in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

10. The method of claim 9 , wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

11. The method of claim 9 , wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

12. The method of claim 11 , wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

13. The method of claim 11 , wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

14. The method of claim 9 , wherein the linear transform-based speaker adaptation uses relevance smoothing of the speech recognition data from the session.

15. A non-transitory computer-readable medium storing computer-readable instructions that, when executed by a processor, cause a device to:
load a user profile for a particular speaker including an initial user feature transform representing speaker adapted speech recognition acoustic models;
perform speech recognition for a session of speech utterances from the particular speaker using the initial user feature transform and a plurality of speaker independent speech recognition acoustic models;
determine a session update transform for a linear transform-based speaker adaptation of the initial user feature transform based on speech recognition data from the session;
produce an updated user feature transform by combining the initial user feature transform and the session update transform; and
store in the user profile the updated user feature transform for subsequent speech recognition with the particular speaker.

16. The non-transitory computer-readable medium of claim 15 , wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed without a bias vector.

17. The non-transitory computer-readable medium of claim 16 , wherein producing the updated user feature transform by combining the initial user feature transform and the session update transform is performed with a bias vector.

18. The non-transitory computer-readable medium of claim 17 , wherein the bias vector is an acoustic observation vector for constrained maximum likelihood linear regression (CMLLR) adaptation.

19. The non-transitory computer-readable medium of claim 17 , wherein the bias vector is a component mean vector for maximum likelihood linear regression (MLLR) adaptation.

20. The non-transitory computer-readable medium of claim 15 , wherein the linear transform-based speaker adaptation uses relevance smoothing of the speech recognition data from the session.

Assignees

• Gollan Christian
• Willett Daniel
• Nuance Communications Inc

Inventors

• Gollan Christian
• Willett Daniel

Classifications

• G10L2015/227

  of the speaker; Human-factor methodology · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G10L15/07

  to the speaker · CPC title

• G10L15/285 Primary

  Memory allocation or algorithm optimisation to reduce hardware requirements · CPC title

• G10L17/06 Primary

  Decision making techniques; Pattern matching strategies · CPC title