Domain adaptation of machine learning models for classification

Patent metadata

Field	Value
Publication number	US-12481833-B2
Application number	US-202217836591-A
Country	US
Kind code	B2
Filing date	Jun 9, 2022
Priority date	Jun 9, 2022
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

The system receives a base machine learning model trained using a generic dataset. For example, the base machine learning model may be an off-the-shelf machine learning based model. The base machine learning model is trained to receive an input and generate a feature vector representing the input. The input may be a natural language expression, an image, or any other type of input. The system receives a domain specific training dataset based on known categories for input values. The system determines an orthogonal transformation for reducing the dimensions of the base machine learning model using on the domain specific training dataset. The system applies the orthogonal transformation to the base machine learning model to obtain a domain specific machine learning model. The system uses the domain specific machine learning model for processing inputs, for example, in a production environment.

First claim

What is claimed is:

1 . A method for determining an intent of natural language expressions used in chatbot conversations conducted via a computing system providing computing services to a plurality of recipients via the Internet, the computing services including access to a relational database system, the method comprising:
receiving a base machine learning model trained using a generic training dataset, the base machine learning model trained to receive a natural language expression and generate a feature vector representing the natural language expression; receiving a domain specific training dataset based on known intents for a chatbot; determining a domain specific machine learning model by applying to the base machine learning model an orthogonal transformation reducing dimensions of the base machine learning model based on the domain specific training dataset, the orthogonal transformation comprising a change of basis via rotation and reflection followed by a truncation of coordinates of the base machine learning model; receiving an input natural language expression used in a chatbot conversation; executing the domain specific machine learning model based on the received natural language expression to obtain a feature vector representing the input natural language expression; comparing the feature vector representing the input natural language expression to one or more stored feature vectors previously mapped to intents; determining an intent for the input natural language expression based on an intent of a matching stored feature vector; and taking an action within the computing system to update one or more values within the relational database system based on the determined intent.

2 . The method of claim 1 , wherein determining an orthogonal transformation comprises:
initializing the orthogonal transformation based on random values; and performing gradient descent based on the domain specific training dataset to modify the orthogonal transformation.

3 . The method of claim 2 , wherein determining an orthogonal transformation comprises:
initializing the orthogonal transformation based on random values; generating a set of triplets based on the domain specific training dataset, wherein a triplet includes an anchor data point, a positive data point that has same intent as the anchor data, and a negative data point that has different intent compared to the anchor data; and modifying the orthogonal transformation to minimize a loss value across the set of triplets.

4 . The method of claim 3 , wherein the loss value is based on:
a first measure of distance between the anchor data point and the positive data point, wherein the loss value increases with increase in the first measure of distance; and a second measure of distance between the anchor data point and the negative data point, wherein the loss value decreases with increase in the second measure of distance.

5 . The method of claim 3 , wherein the loss value is based on a measure of difference between (1) a product of the orthogonal transformation and a transpose of the orthogonal transformation and (2) an identity transformation.

6 . The method of claim 1 , wherein the action comprises one or more of:
accessing data in a data store, interacting with an external system, or performing a transaction.

7 . The method of claim 1 , wherein the size of the generic training dataset is greater than the size of the domain specific training dataset.

8 . The method of claim 1 , wherein the domain specific training dataset comprises, for one or more categories, a plurality of input values mapped to each category.

9 . A non-transitory computer-readable storage medium storing instructions that when executed by one or more computer processors, causes the one or more computer processors to perform a method for determining an intent of natural language expressions used in chatbot conversations conducted via a computing system providing computing services to a plurality of recipients via the Internet, the computing services including access to a relational database system, the method comprising:
receiving a base machine learning model trained using a generic training dataset, the base machine learning model trained to receive a natural language expression and generate a feature vector representing the natural language expression; receiving a domain specific training dataset based on known intents for a chatbot; determining a domain specific machine learning model by applying to the base machine learning model an orthogonal transformation reducing dimensions of the base machine learning model based on the domain specific training dataset, the orthogonal transformation comprising a change of basis via rotation and reflection followed by a truncation of coordinates of the base machine learning model; receiving an input natural language expression used in a chatbot conversation; executing the domain specific machine learning model based on the received natural language expression to obtain a feature vector representing the input natural language expression; comparing the feature vector representing the input natural language expression to one or more stored feature vectors previously mapped to intents; determining an intent for the input natural language expression based on an intent of a matching stored feature vector; and taking an action within the computing system to update one or more values within the relational database system based on the determined intent.

10 . The non-transitory computer-readable storage medium of claim 9 , wherein instructions for determining an orthogonal transformation cause the one or more computer processors to perform steps comprising:
initializing the orthogonal transformation based on random values; and performing gradient descent based on the domain specific training dataset to modify the orthogonal transformation.

11 . The non-transitory computer-readable storage medium of claim 10 , wherein instructions for determining an orthogonal transformation cause the one or more computer processors to perform steps comprising:
initializing the orthogonal transformation based on random values; generating a set of triplets based on the domain specific training dataset, wherein a triplet includes an anchor data point, a positive data point that has same intent as the anchor data, and a negative data point that has different intent comparted to the anchor data; and modifying the orthogonal transformation to minimize a loss value across the set of triplets.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein the loss value is based on:
a first measure of distance between the anchor data point and the positive data point, wherein the loss value increases with increase in the first measure of distance; and a second measure of distance between the anchor data point and the negative data point, wherein the loss value decreases with increase in the second measure of distance.

13 . The non-transitory computer-readable storage medium of claim 11 , wherein the loss value is based on a measure of difference between (1) a product of the orthogonal transformation and a transpose of the orthogonal transformation and (2) an identify transformation.

14 . The non-transitory computer-readable storage medium of claim 9 , wherein the action comprises one or more of:
accessing data in a data store, interacting with an external system, or performing a transaction.

15 . A computing system for determining an intent of natural language expressions used in chatbot conversation

Assignees

• Salesforce Inc

Inventors

• Alexander Zachary
• Mehrotra Shubham

Classifications

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• G06N20/00

  Machine learning · CPC title