Implementing a classification model for recognition processing

Patent metadata

Field	Value
Publication number	US-10529318-B2
Application number	US-201514815564-A
Country	US
Kind code	B2
Filing date	Jul 31, 2015
Priority date	Jul 31, 2015
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A method, system, and computer program product for learning a recognition model for recognition processing. The method includes preparing one or more examples for learning, each of which includes an input segment, an additional segment adjacent to the input segment and an assigned label. The input segment and the additional segment are extracted from an original training data. A classification model is trained, using the input segment and the additional segment in the examples, to initialize parameters of the classification model so that extended segments including the input segment and the additional segment are reconstructed from the input segment. Then, the classification model is tuned to predict a target label, using the input segment and the assigned label in the examples, based on the initialized parameters. At least a portion of the obtained classification model is included in the recognition model.

First claim

What is claimed is:

1. A method for learning a recognition model for recognition processing, the method comprising:
preparing one or more examples for learning, each of which includes an input segment, an additional segment adjacent to the input segment and an assigned label, the input segment and the additional segment being extracted from an original training data;
training a classification model, using a processor, using the input segment and the additional segment in the examples to initialize parameters of the classification model to provide initialized parameters so that extended segments, including the input segment and the additional segment, are reconstructed from the input segment without use of further segments to provide reconstructed extended segments; and
tuning the classification model to predict a target label, using the input segment and the assigned label in the examples, based on the initialized parameters, at least a portion of the classification model being included in the recognition model.

2. The method of claim 1 , wherein the recognition model includes all portions of the classification model, the classification model estimating posterior probabilities over targets.

3. The method of claim 1 , wherein the recognition model uses the at least portion of the classification model as a feature extraction model for a feature extractor and includes a post-stage recognition model, the feature extractor outputting estimated target probabilities or activations of an internal layer of the classification model as features for the post-stage recognition model, the post-stage recognition model estimating posterior probabilities over targets based on the features.

4. The shod of claim 1 , wherein a size of input for recognition to the classification model is equal to a size of the input segment for learning.

5. The method of claim 1 , wherein information outside the input segment is subsumed in the classification model by treating the input segment as input and the extended segments as prediction of the input during the training.

6. The method of claim 1 , wherein the training includes:
optimizing forward mapping parameters and reverse mapping parameters of a layer in the classification model such that a discrepancy between the extended segments and the reconstructed extended segments from the input segment is minimized, the reverse mapping parameters being discarded in response to stacking the layer within the classification model.

7. The method of claim 6 , wherein a regularization term is added to a loss function measuring the discrepancy, the regularization term penalizing larger values of the reverse mapping parameters so as to subsume more information h forward mapping parameters than the reverse mapping parameters.

8. The method of claim 1 , wherein the classification model includes a deep neural network having one or more hidden layers between an input layer for the input segment and an output layer for targets, the training is included in an unsupervised pre-training process that stacks the one or more hidden layers, the input layer, and the output layer with initializing parameters, the tuning is a fine-tuning process that discriminatively updates the parameters of the layers, the deep neural network is incorporated into a hidden Markov model (HMM) and the targets of the classification model are HMM states.

9. The method of claim 1 , wherein the original training data is acoustic data, the input segment is n-frame acoustic features, the extended segment is n+m-frame acoustic features, and the additional segment is m-frame acoustic features preceding and/or succeeding the n-frame acoustic features.

10. The method of claim 1 , wherein the original training data is image data, the input segment is a x*y pixel patch, the extended segment is (x+a)*(y+b) pixel patch, and the additional segment is a (b*x+a*y+a*b) pixel patch surrounding the x*y pixel patch.

11. The method of claim 1 , wherein at least one of the training and the tuning of the classification model is performed in a cloud corrupting environment.

12. A computer system for learning a recognition model for recognition processing by executing program instructions tangibly stored in a memory, the computer system comprising:
a processor in communication with the memory, wherein the computer system is configured to:
prepare one or more examples for learning, each of which includes an input segment, an additional segment adjacent to the input segment and an assigned label, the input segment and the additional segment being extracted from an original training data;
train a classification model, using the input segment and the additional segment in the examples to initialize parameters of the classification model to provide initialized parameters so that extended segments, including the input segment and the additional segment, are reconstructed from the input segment without use of further segments to provide reconstructed extended segments; and
tune the classification model to predict a target label, using the input segment and the assigned label in the examples, based on the initialized parameters, at least a portion of the classification model being included in the recognition model.

13. The computer system of claim 12 , wherein the recognition model includes a whole of the classification model, the classification model estimating posterior probabilities over targets.

14. The computer system of claim 12 , wherein the recognition model uses the at least a portion of the classification model as a feature extraction model for a feature extractor and includes a post-stage recognition model, the feature extractor outputting estimated target probabilities or activations of an internal layer of the classification model as features for the post-stage recognition model, the post-stage recognition model estimating posterior probabilities over targets based on the features.

15. The computer system of claim 12 , wherein a size of input for recognition to the classification model is equal to a size of the input segment for learning.

16. The computer system of claim 12 , wherein information outside the input segment is subsumed in the classification model by treating the input segment as input and the extended segments as prediction of the input during the training.

17. The computer system of claim 12 , wherein the computer system is further configured to:
optimize forward mapping parameters and reverse mapping parameters of a layer in the classification model such that a discrepancy between the extended segments and the reconstructed extended segments from the input segment is minimized, the reverse mapping parameters being discarded in response to stacking the layer within the classification model.

18. The computer system of claim 17 , wherein a regularization term is added to a loss function measuring the discrepancy, the regularization term penalizing larger value of the reverse mapping parameters so as to subsume more information into the forward mapping parameters than the reverse mapping parameters.

19. The computer system of claim 12 , wherein the computer system is provided in a cloud computing environment.

20. A computer program product for learning a recognition model for recognition processing, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
preparing one or more examples for le

Assignees

• IBM

Inventors

• Kurata Gakuto

Classifications

• G06N20/00

  Machine learning · CPC title

• G10L15/16 Primary

  using artificial neural networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title