Speech recognition method and apparatus

Patent metadata

Field	Value
Publication number	US-10930268-B2
Application number	US-201916244397-A
Country	US
Kind code	B2
Filing date	Jan 10, 2019
Priority date	May 31, 2018
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

Disclosed is a speech recognition method and apparatus, wherein the apparatus acquires first outputs from sub-models in a recognition model based on a speech signal, acquires a second output including values corresponding to the sub-models from a classification model based on the speech signal, and recognizes the speech signal based on the first outputs and the second output.

First claim

What is claimed is:

1. A speech recognition method comprising:
acquiring first outputs from sub-models in a recognition model based on a speech signal;
acquiring, based on the speech signal, a second output comprising respective probabilities that the speech signal belongs to each of the sub-models from a classification model; and
recognizing the speech signal based on a weighted sum of the first outputs and the second output,
wherein the sub-models comprise models for estimating pronunciations classified into groups based on a similarity of pronunciation for each of the groups, and
wherein the classification model is configured to estimate a probability or weight associated with each of the sub-models and nodes in an output layer of the classification model correspond to each of the sub-models.

2. The speech recognition method of claim 1 , wherein the groups are classified based on any one or any combination of dialects, regions, and races in a language.

3. The speech recognition method of claim 1 , wherein the sub-models are models for estimating pronunciations for each of the users.

4. The speech recognition method of claim 1 , wherein the recognition model comprises a neural network for estimating a pronunciation of the speech signal, and
the sub-models each include learning hidden unit contributions (LHUCs) or layers trained independently of one another in the neural network.

5. The speech recognition method of claim 4 , wherein the sub-models share at least one layer in the neural network.

6. The speech recognition method of claim 5 , wherein a feature acquired from a layer shared by the sub-models is applied to the trained layers.

7. The speech recognition method of claim 1 , wherein the acquiring of the second output comprises:
acquiring a feature from a layer in the recognition model; and
acquiring the second output by applying the acquired feature to the classification model.

8. The speech recognition method of claim 1 , wherein the acquiring of the second output comprises:
generating a feature suitable for an input layer of the classification model based on the speech signal; and
acquiring the second output by applying the generated feature to the classification model.

9. The speech recognition method of claim 1 , wherein the recognizing of the speech signal comprises:
generating a third output based on the weighted sum between the first outputs and the respective probabilities included in the second output; and
estimating a pronunciation of the speech signal based on the third output.

10. The speech recognition method of claim 1 , wherein the acquiring of the second output comprises:
applying a bias to the probabilities included in the second output based on a context associated with the speech signal, and
the context comprises any one or any combination of a location and a language of a keyboard of a device to which the speech signal is applied.

11. The speech recognition method of claim 1 , wherein an acoustic model comprising the recognition model and the classification model are connected to a language model on an end-to-end basis, and
the recognizing of the speech signal comprises:
recognizing a word or a sentence of the speech signal based on the first outputs and the second output.

12. The speech recognition method of claim 1 , wherein the respective probabilities comprise weights corresponding to each of the sub-models.

13. The speech recognition method of claim 1 , wherein the recognition model is trained to recognize a language of the users, and the sub-models are trained to recognize languages corresponding to groups of the users.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

15. A speech recognition method comprising:
generating an input feature of a recognition model including sub-models based on a speech signal;
acquiring probabilities that the speech signal belongs to the respective sub-models from a classification model based on the speech signal;
generating a second input feature by applying the probabilities to the input feature; and
recognizing the speech signal by applying the second input feature to the recognition model,
wherein the sub-models comprise models for estimating pronunciations classified into groups based on a similarity of pronunciation for each of the groups, and
wherein the classification model is configured to estimate a probability or weight associated with each of the sub-models and nodes in an output layer of the classification model correspond to each of the sub-models.

16. The speech recognition method of claim 15 , wherein an input layer of the recognition model comprises nodes corresponding to the probabilities.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 15 .

18. A speech recognition apparatus comprising:
a processor configured
to acquire first outputs from sub-models in a recognition model based on a speech signal,
to acquire, based on the speech signal, a second output comprising respective probabilities that the speech signal belongs to each of the sub-models from a classification model, and
to recognize the speech signal based on a weighted sum of the first outputs and the second output,
wherein the sub-models comprise models for estimating pronunciations classified into groups based on a similarity of pronunciation for each of the groups, and
wherein the classification model is configured to estimate a probability or weight associated with each of the sub-models and nodes in an output layer of the classification model correspond to each of the sub-models.

19. A training apparatus for speech recognition, the apparatus comprising:
a processor configured
to train a recognition model comprising sub-models based on first training speech signals,
to train sub-models based on second training speech signals corresponding to the sub-models, and
to train a classification model that generates outputs corresponding to the sub-models based on the second training speech signals.

20. A speech recognition method comprising:
generating a first input feature for a recognition model comprising sub-models based on a speech signal;
acquiring respective probabilities of the speech signal belonging to each of the sub-models from a classification model based on the speech signal;
generating a second input feature for the recognition model based on applying the probabilities to the first input feature; and
recognizing the speech signal based on an output generated, in response to the second input feature being applied to the classification model,
wherein the sub-models comprise models for estimating pronunciations classified into groups based on a similarity of pronunciation for each of the groups, and
wherein the classification model is configured to estimate a probability or weight associated with each of the sub-models and nodes in an output layer of the classification model correspond to each of the sub-models.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Yoo Sang Hyun
• Mun Minyoung
• SONG Inchul

Classifications

• G10L15/187 Primary

  Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams · CPC title

• G10L15/16 Primary

  using artificial neural networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title