Teacher and student learning for constructing mixed-domain model

Patent metadata

Field	Value
Publication number	US-11416741-B2
Application number	US-201816003790-A
Country	US
Kind code	B2
Filing date	Jun 8, 2018
Priority date	Jun 8, 2018
Publication date	Aug 16, 2022
Grant date	Aug 16, 2022

Abstract

A technique for constructing a model supporting a plurality of domains is disclosed. In the technique, a plurality of teacher models, each of which is specialized for different one of the plurality of the domains, is prepared. A plurality of training data collections, each of which is collected for different one of the plurality of the domains, is obtained. A plurality of soft label sets is generated by inputting each training data in the plurality of the training data collections into corresponding one of the plurality of the teacher models. A student model is trained using the plurality of the soft label sets.

First claim

What is claimed is:

1. A computer-implemented method for constructing a model supporting a plurality of domains, the method comprising:
preparing a plurality of teacher models, each teacher model being specialized for different one of the plurality of the domains;
obtaining a plurality of training data collections, each of the plurality of training data collections being collected for a different one of the plurality of the domains;
inputting training data from each of the plurality of training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and
training a student model using the plurality of the soft label sets.

2. The method of claim 1 , wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

3. The method of claim 2 , wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the preparing of the plurality of the teacher models comprises:
training each teacher model using matched features extracted by the matched feature extractor from teacher training data for the corresponding one of the plurality of the domains.

4. The method of claim 2 , wherein the unified feature extractor of the student model extracts an unified feature from an input signal in any one of the plurality of the domains by unifying physical meanings of features between the plurality of the domains.

5. The method of claim 4 , wherein the unified feature extractor of the student model includes a hybrid normalization parameter set used in common for the plurality of the domains.

6. The method of claim 4 , wherein a first unified feature extracted for a first domain has a plurality of elements and a second unified feature extracted for a second domain has a part of elements corresponding to a part of the elements for the first domain and a remaining part of elements corresponding to a remaining part of the elements for the first domain and having a predetermined value.

7. The method of claim 2 , wherein the training of the student model comprises:
extracting an unified feature by the unified feature extractor from training data in each of the plurality of training data collections; and
using the unified feature and a soft label associated with the unified feature as an input to the student model and privileged information, respectively.

8. The method of claim 2 , wherein the plurality of the teacher models and the student model are acoustic models and the plurality of the domains has difference in sampling condition of an input speech signal.

9. The method of claim 2 , wherein the plurality of the teacher models and the student model are image processing models and the plurality of the domains has difference in color mode of an input image signal.

10. The method of claim 1 , wherein the student model is a neural network based model.

11. A computer system for constructing a model supporting a plurality of domains, the computer system comprising:
a memory storing program instructions;
a processing circuitry in communications with the memory for executing the program instructions, wherein the program instructions are configured to:
prepare a plurality of teacher models, wherein each teacher model is specialized for different one of the plurality of the domains;
obtain a plurality of training data collections, wherein each of the plurality of training data collections is collected for a different one of the plurality of the domains;
input training data from each of the plurality of training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and
train a student model using the plurality of the soft label sets.

12. The computer system of claim 11 , wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

13. The computer system of claim 12 , wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the processing circuitry is further configured to:
train each teacher model using matched features extracted by the matched feature extractor from teacher training data for the corresponding one of the plurality of the domains to prepare the plurality of the teacher models.

14. The computer system of claim 12 , wherein the unified feature extractor of the student model extracts an unified feature from an input signal in any one of the plurality of the domains by unifying physical meanings of features between the plurality of the domains.

15. The computer system of claim 14 , wherein the unified feature extractor of the student model includes a hybrid normalization parameter set used in common for the plurality of the domains.

16. The computer system of claim 12 , wherein the processing circuitry is further configured to:
extract an unified feature by the unified feature extractor from training data in each of the plurality of training data collections; and
use the unified feature and a soft label associated with the unified feature as an input to the student model and privileged information, respectively.

17. The computer system of claim 12 , wherein the plurality of the teacher models and the student model are acoustic models and the plurality of the domains has difference in sampling condition of an input speech signal.

18. A computer program product constructing a model supporting a plurality of domains, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
preparing a plurality of teacher models, each teacher model being specialized for different one of the plurality of the domains;
obtaining a plurality of training data collections, each of the plurality of training data collections being collected for a different one of the plurality of the domains;
inputting training data from each of the plurality of the training data collections into a corresponding one of the plurality of the teacher models to generate a plurality of soft label sets; and
training a student model using the plurality of the soft label sets.

19. The computer program product of claim 18 , wherein each teacher model is connected to a matched feature extractor for corresponding one of the plurality of the domains and the student model is connected to a unified feature extractor, the unified feature extractor being common at least partially to the plurality of the domains.

20. The computer program product of claim 19 , wherein the matched feature extractor of each teacher model extracts a matched feature from an input signal in the corresponding one of the plurality of the domains and the preparing of the plurality of the teacher models comprises:
training each teacher model using matched features extracted by the matched feature extractor from teacher training data for the correspondi

Assignees

• IBM

Inventors

• Fukuda Takashi
• Ichikawa Osamu
• Thomas Samuel
• Ramabhadran Bhuvana

Classifications

• G06V10/7753

  Incorporation of unlabelled data, e.g. multiple instance learning [MIL] · CPC title

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/32

  Normalisation of the pattern dimensions · CPC title

• G06V10/811

  the classifiers operating on different input data, e.g. multi-modal recognition · CPC title