Method and apparatus with a personalized speech recognition model

Patent metadata

Field	Value
Publication number	US-11037552-B2
Application number	US-201815989366-A
Country	US
Kind code	B2
Filing date	May 25, 2018
Priority date	Dec 29, 2017
Publication date	Jun 15, 2021
Grant date	Jun 15, 2021

Abstract

A method and apparatus for personalizing a speech recognition model is disclosed. The apparatus may obtain feedback data that is a result of recognizing a first speech input of a user using a trained speech recognition model, determine whether to update the speech recognition model based on the obtained feedback data, and selectively update, dependent on the determining, the speech recognition model based on the feedback data.

First claim

What is claimed is:

1. A processor-implemented speech recognition method, the method comprising:
obtaining feedback data that is a result of recognizing a first speech input of a user using a trained neural network speech recognition model;
determining whether to update the speech recognition model based on the obtained feedback data; and
selectively, dependent on the determining, updating the speech recognition model based on the feedback data,
wherein the determining of whether to update the speech recognition model comprises:
obtaining a temporary speech recognition model obtained by training the speech recognition model, including personalizing at least an acoustic model of the speech recognition model, based on the feedback data;
calculating a first error rate of the temporary speech recognition model; and
determining whether to update the speech recognition model based on the first error rate and a calculated second error rate of the speech recognition model.

2. The method of claim 1 , further comprising performing recognition of input speech using the updated speech recognition model provided the input speech.

3. The method of claim 1 , wherein the obtaining of the feedback data comprises:
receiving a guide text from the user;
receiving the first speech input corresponding to the received guide text; and
obtaining the feedback data based on the received guide text and the received first speech input.

4. The method of claim 1 , wherein the obtaining of the feedback data comprises:
receiving the first speech input of the user;
receiving, from the user, an answer text corresponding to the received first speech input; and
obtaining the feedback data based on the received first speech input and the received answer text.

5. The method of claim 1 , wherein the obtaining of the feedback data comprises:
receiving the first speech input;
generating a guide text corresponding to the received first speech input;
receiving a second speech input of the user corresponding to the generated guide text; and
obtaining the feedback data based on the generated guide text and the received second speech input.

6. The method of claim 1 , wherein the obtaining of the temporary speech recognition model comprises:
training the speech recognition model based on one or more sets of feedback data accumulated since an initial point in time and including the feedback data.

7. The method of claim 1 , wherein the obtaining of the temporary speech recognition model comprises:
training the speech recognition model based on one or more sets of feedback data accumulated only since a point in time after an initial point in time and including the feedback data, where the point in time, after the initial point in time, is a time or period of time in which feedback data was previously generated subsequent to feedback data generated with respect the initial point in time.

8. The method of claim 1 , wherein the obtaining of the temporary speech recognition model comprises:
training the speech recognition model based on the feedback data and training data representing speech of multiple individuals.

9. The method of claim 1 , wherein the updating of the speech recognition model comprises:
in response to the first error rate being less than the second error rate, replacing the speech recognition model with the temporary speech recognition model.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1 .

11. A speech recognition apparatus, the apparatus comprising:
at least one memory configured to store a trained neural network speech recognition model; and
one or more processors configured to:
obtain feedback data that is a result of recognizing a first speech input of a user using the speech recognition model;
determine whether to update the speech recognition model based on the feedback data; and
selectively, dependent on the determining, update the speech recognition model based on the feedback data,

wherein, for the determining of whether to update the speech recognition model, the one or more processors are further configured to:
obtain a temporary speech recognition model obtained by training the speech recognition model, including personalizing at least an acoustic model of the speech recognition model, based on the feedback data;
calculate a first error rate of the temporary speech recognition model; and
determine whether to update the speech recognition model based on the first error rate and a calculated second error rate of the speech recognition model.

12. The apparatus of claim 11 , wherein the one or more processors are further configured to perform recognition of input speech using the updated speech recognition model provided the input speech.

13. The apparatus of claim 11 , wherein the memory stores parameters of the speech recognition model, and the re-training of the speech recognition model includes generating a personalized neural network speech recognition model by at least adjusting parameters of the acoustic model of the speech recognition model based on the feedback data.

14. The apparatus of claim 11 , wherein, for the obtaining of the feedback data, the one or more processors are configured to:
receive a guide text from the user;
receive the first speech input corresponding to the received guide text; and
obtain the feedback data based on the received guide text and the received first speech input.

15. The apparatus of claim 11 , wherein, for the obtaining of the feedback data, the one or more processors are configured to:
receive the first speech input of the user;
receive, from the user, an answer text corresponding to the received first speech input; and
obtain the feedback data based on the received first speech input and the received answer text.

16. The apparatus of claim 11 , wherein, for the obtaining of the feedback data, the one or more processors are configured to:
receive the first speech input;
generate a guide text corresponding to the received first speech input;
receive a second speech input of the user corresponding to the generated guide text; and
obtain the feedback data based on the generated guide text and the received second speech input.

17. The apparatus of claim 11 , wherein the one or more processors are further configured to:
train the speech recognition model based on one or more sets of feedback data accumulated since an initial point in time and including the feedback data.

18. The apparatus of claim 11 , wherein the one or more processors are further configured to:
train the speech recognition model based on one or more sets of feedback data accumulated only since a point in time after an initial point in time and including the feedback data, where the point in time, after the initial point in time, is a time or period of time in which feedback data was previously generated subsequent to feedback data generated with respect the initial point in time.

19. The apparatus of claim 11 , wherein the one or more processors are further configured to:
train the speech recognition model based on the feedback data and training data representing speech of multiple individuals.

20. The apparatus of claim 11 , further comprising one or more memories storing instructions, which when executed by the one or more processors, cause the one or more processors to perform the obtaining of the feedback data, the determining of whether to up

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kwon Ki Soo
• Mun Minyoung
• Yoo Sanghyun

Classifications

• G10L2015/225

  Feedback of the input speech · CPC title

• G10L15/18

  using natural language modelling · CPC title

• G10L15/07 Primary

  to the speaker · CPC title

• G10L2015/0635

  updating or merging of old and new templates; Mean values; Weighting · CPC title

• G10L15/32

  Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems · CPC title