Transfer learning techniques for disparate label sets

Patent metadata

Field	Value
Publication number	US-11062228-B2
Application number	US-201514792269-A
Country	US
Kind code	B2
Filing date	Jul 6, 2015
Priority date	Jul 6, 2015
Publication date	Jul 13, 2021
Grant date	Jul 13, 2021

Abstract

Examples of the present disclosure describe systems and methods of transfer learning techniques for disparate label sets. In aspects, a data set may be accessed on a server device. The data set may comprise labels and word sets associated with the labels. The server device may induce label embedding within the data set. The embedded labels may be represented by multi-dimensional vectors that correspond to particular labels. The vectors may be used to construct label mappings for the data set. The label mappings may be used to train a model to perform domain adaptation or transfer learning techniques. The model may be used to provide results to a statement/query or to train a different model.

First claim

What is claimed is:

1. A system comprising:
at least one processor; and
memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising:
accessing a first set of labeled data associated with at least a first domain, wherein the first set of labeled data comprises one or more labels and corresponding data values;
transforming the first set of labeled data into a set of vectors, wherein the set of vectors includes at least a first vector comprising a first label and a second vector comprising a second label;
using one or more clustering techniques to identify a commonality between the first label and the second label, wherein the commonality indicates the first label and the second label belong to a shared cluster category;
based at least on the identified commonality between the first label and the second label, generating a coarse label set comprising a third label representing the shared cluster category, wherein the third label represents an abstraction of the first label and the second label;
using the coarse label set, training a model for a second domain to perform transfer learning techniques associated with natural language understanding, wherein the transfer learning techniques comprise adapting data in the first domain to the second domain and training the model using a union of data from the first domain and the second domain, wherein the second domain is different from the first domain; and
using the trained model, mapping a first coarse label associated with the coarse label set to a fine label associated with the second domain.

2. The system of claim 1 , further comprising: receiving the labeled data from a received input, wherein the labeled data comprises query data associated with the received input.

3. The system of claim 2 , further comprising: using the model to provide a result set of a query to the user.

4. The system of claim 1 , wherein transforming the first set of labeled data comprises applying canonical correlation analysis (CCA) to the first set of labeled data.

5. The system of claim 1 , wherein generating the coarse label set comprises aggregating at least the first vector and the second vector using at least one technique selected from the group consisting of k-means clustering, spectral clustering, affinity propagation, mean-shift, Ward hierarchical clustering, agglomerative clustering, DB SCAN, Gaussian mixtures, and Birch clustering.

6. The system of claim 1 , wherein the coarse label set is generalized from the at least two vectors.

7. The system of claim 1 , wherein training the model comprises identifying one or more labels that are semantically related to the coarse label set.

8. The system of claim 6 , further comprising predicting labels for a target domain using the identified one or more labels.

9. The system of claim 1 , wherein the second label shares the commonality of at least the first vector and the second vector.

10. A system for mapping disparate label sets, the system comprising:
at least one processor; and
memory coupled to the at least one processor, the memory comprising computer executable instructions that, when executed by the at least one processor, performs a method comprising:
accessing a set of labeled data, wherein the set of labeled data comprises data from a first domain and data from a second domain;
transforming the set of labeled data into a set of vectors, wherein each of the set of vectors is associated with one or more labels;
identifying, in the set of vectors, a first vector comprising labeled data from the first domain and a second vector comprising labeled data from the second domain, wherein a first label of the first vector shares a commonality with a second label of the second vector, wherein the commonality indicates the first label and the second label belong to a shared cluster category;
constructing label mappings to map the first vector to the second vector based on the commonality; and
using at least the second vector, training a model to perform transfer learning techniques associated with natural language understanding, wherein the transfer learning techniques comprise adapting data in the first domain to the second domain and training the model using a union of data from the first domain and the second domain, the second domain being different from the first domain, and wherein training the model comprises duplicating the set of vectors and conjoining vectors in the duplicated set of vectors with a domain indicator.

11. The system of claim 10 , further comprising: receiving the labeled data from a user, wherein the labeled data comprises query data associated with the user.

12. The system of claim 10 , wherein transforming the first set of labeled data comprises applying canonical correlation analysis (CCA) to the first set of labeled data.

13. The system of claim 10 , wherein the commonality represents that the first vector is a nearest match in the data from the first domain to the second vector, wherein the nearest match is determined using a k-nearest neighbor algorithm.

14. The system of claim 10 , wherein mapping comprises generating a bijective mapping between the first vector and the second vector.

15. The system of claim 10 , further comprising: using the model to generate a result set.

16. A computer-implemented method for mapping disparate label sets, the method comprising:
accessing, on a device, a first set of labeled data associated with at least a first domain, wherein the first set of labeled data comprises one or more labels and corresponding data values;
transforming the first set of labeled data into a set of vectors;
identifying a commonality between a first label in a first vector from the set of vectors and a second label in a second vector from the set of vectors, wherein the commonality indicates the first label and the second label belong to a shared cluster category;
based on the identified commonality, generating a coarse label set using at least the first vector and the second vector, wherein the coarse label set represents an abstraction of the first label and the second label;
using the coarse label set to train a model for a second domain to perform transfer learning techniques associated with natural language understanding, wherein the transfer learning techniques comprise adapting data in the first domain to the second domain and training the model using a union of data from the first domain and the second domain, wherein the second domain is different from the first domain; and
using the trained model, mapping the one or more labels in the coarse label set to a third label associated with a second set of labeled data.

17. The computer-implemented method of claim 16 , further comprising:
receiving the labeled data from a user, wherein the labeled data comprises query data associated with the user;
using the model to generate a result set from the query data; and
providing the result set to the user.

18. The computer-implemented method of claim 16 , wherein transforming the first set of labeled data comprises applying canonical correlation analysis (CCA) to the first set of labeled data.

19. The system of claim 1 , wherein the first domain is associated with a schema specifying at least one of an intent, a slot, or metadata within the set of labeled data.

20. The system of claim 1 , wherein the trained model is used to make predictions on labels within the secon

Assignees

• Microsoft Technology Licensing Llc
• Microsoft Technoiogy Licensing Llc

Inventors

• Kim Young-Bum
• Sarikaya Ruhi

Classifications

• G06N20/00 Primary

  Machine learning · CPC title