Unsupervised learning system and method for performing weighting for improvement in speech recognition performance and recording medium for performing the method

Patent metadata

Field	Value
Publication number	US-11164565-B2
Application number	US-201916561651-A
Country	US
Kind code	B2
Filing date	Sep 5, 2019
Priority date	Jul 31, 2019
Publication date	Nov 2, 2021
Grant date	Nov 2, 2021

Abstract

A learning system and method for updating recognition performance by assigning weights according to a confidence level of data are discussed. The unsupervised learning system includes a memory configured to store speech data received from a server that performs speech recognition; and a processor configured to measure confidence levels of pieces of learnable data stored in the memory and classify the pieces of learnable data into learning data and adaptation data, generate a learning model by performing unsupervised learning on the learning data, generate an adaption model using the adaptation data, and evaluate speech recognition performance for the learning model and the adaptation model, wherein the processor is configured to assign weights by applying the measured confidence levels to the learning model and the adaptation model and update recognition performance with the learning model and the adaptation model to which the weights are applied.

First claim

What is claimed is:

1. A unsupervised learning apparatus for performing weighting for improvement in speech recognition performance, the apparatus comprising:
a memory configured to store speech data provided from a server that performs speech recognition; and
a processor configured to:
measure confidence levels of pieces of learnable data stored in the memory and classify the pieces of learnable data into learning data and adaptation data, according to the measured confidence levels,
generate a learning model by performing unsupervised learning on the learning data,
generate an adaptation model using the adaptation data, and
evaluate recognition performance for each of the learning model and the adaptation model,
wherein the processor is configured to assign weights by applying the measured confidence levels to the learning model and the adaptation model and update the recognition performance with the learning model and the adaptation model to which the weights are applied, and
wherein the processor is configured to:
calculate new learning data or new adaptation data by applying weights according to the confidence levels to the learning data or the adaptation data,
generate the learning model through the new learning data, and
generate the adaptation model through the new adaptation data.

2. The unsupervised learning apparatus of claim 1 , wherein the processor is configured to:
classify the learnable data into the learning data when the confidence level of the learnable data is greater than or equal to a reference confidence level, and
classify the learnable data into the adaptation data when the confidence level of the learnable data is less than the reference confidence level.

3. The unsupervised learning apparatus of claim 1 , wherein the processor is configured to:
select N pieces of data, each of which a hidden Markov model-state entropy is greater than a reference entropy, among learning data with a confidence level greater than or equal to the reference confidence level, where N is a number,
perform unsupervised learning by using the selected N pieces of data and previously-stored seed data, and
generate the learning model according to a result of the performance of the unsupervised learning.

4. The unsupervised learning apparatus of claim 2 , wherein the processor is configured to generate the adaptation model using a generative adversarial network for adaptation data with a confidence level less than the reference confidence level.

5. The unsupervised learning apparatus of claim 1 , further comprising:
a performance evaluation model configured to evaluate performance of the learning model and the adaptation model,
wherein the performance evaluation model
measures a first performance evaluation value indicating a number of successes of speech recognition in which the learning model is applied to logging speech data and a second performance evaluation value indicating a number of successes of speech recognition in which the adaptation model is applied to logging speech data, and
selects a model corresponding to a larger performance evaluation value of the first performance evaluation value and the second performance evaluation value among the learning model and the adaptation model.

6. The unsupervised learning apparatus of claim 5 , wherein the processor is configured to:
compare a performance evaluation value of the selected model with a performance evaluation value of an acoustic model stored previously, and
update the acoustic model with the selected model when the performance evaluation value of the selected model is larger than the performance evaluation value of the acoustic model.

7. The unsupervised learning apparatus of claim 1 , wherein the processor is configured to update a performance evaluation model with the learning model or the adaptation model to which the weights are applied.

8. A unsupervised learning method for performing weighting for improvement in speech recognition performance, the method comprising:
measuring confidence levels of pieces of learnable data of speech data received from a server that performs speech recognition and stored;
classifying the pieces of learnable data according to the measured confidence levels into learning data or adaptation data;
generating a learning model by performing unsupervised learning on the learning data and generating an adaptation model using the adaptation data; and
evaluating speech recognition performance for the learning model and the adaptation model,
wherein the unsupervised learning method further comprises:
assigning weights by applying the measured confidence levels to the learning model and the adaptation model; and
updating the speech recognition performance with the learning model or the adaptation model to which the weights are applied.

9. The unsupervised learning method of claim 8 , wherein the evaluating of the speech recognition performance includes
measuring a first performance evaluation value indicating a number of successes of speech recognition in which the learning model is applied to logging speech data and a second performance evaluation value indicating a number of successes of speech recognition in which the adaptation model is applied to logging speech data, and
selecting a model corresponding to a larger performance evaluation value of the first performance evaluation value and the second performance evaluation value among the learning model and the adaptation model.

10. The unsupervised learning method of claim 9 , further comprising:
comparing a performance evaluation value of the selected model with a performance evaluation value of an acoustic model stored previously; and
updating the acoustic model with the selected model when the performance evaluation value of the selected model is larger than the performance evaluation value of the acoustic model.

11. The unsupervised learning method of claim 8 , wherein the classifying of the pieces of learnable data includes
classifying the learnable data into the learning data when the measured confidence level is greater than or equal to a reference confidence level, and
classifying the learnable data into the adaptation data when the measured confidence level is less than the reference confidence level.

12. The unsupervised learning method of claim 8 , wherein the generating of the learning model includes
selecting N pieces of data of which a hidden Markov model-state entropy is greater than a reference entropy, among learning data with a confidence level greater than or equal to the reference confidence level, where N is a number,
performing unsupervised learning by using the selected N pieces of data and previously-stored seed data, and
generating the learning model according to a result of performance of the unsupervised learning.

13. The unsupervised learning method of claim 8 , wherein the generating of the adaptation model includes generating the adaptation model using a generative adversarial network for adaptation data with a confidence level less than the reference confidence level.

Assignees

• Lg Electronics Inc

Inventors

• Lee Jeehye

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G10L15/144 Primary

  Training of HMMs · CPC title