Routing text classifications within a cross-domain conversational service

Patent metadata

Field	Value
Publication number	US-11270077-B2
Application number	US-201916411076-A
Country	US
Kind code	B2
Filing date	May 13, 2019
Priority date	May 13, 2019
Publication date	Mar 8, 2022
Grant date	Mar 8, 2022

Abstract

A computing device receives a natural language input from a user. The computing device routes the natural language input from an active domain node of multiple domain nodes of a multi-domain context-based hierarchy to a leaf node of the domain nodes by selecting a parent domain node in the hierarchy until an off-topic classifier labels the natural language input as in-domain and then selecting a subdomain node in the hierarchy until an in-domain classifier labels the natural language input with a classification label, each of the plurality of domain nodes comprising a respective off-topic classifier and a respective in-domain classifier trained for a respective domain node. The computing device outputs the classification label determined by the leaf node.

First claim

What is claimed is:

1. A method, comprising:
receiving, by a computing device, a natural language input from a user;
routing, by the computing device, the natural language input from an active domain node of a plurality of domain nodes of a multi-domain context-based hierarchy to a leaf node of the plurality of domain nodes by selecting a parent domain node in the hierarchy until an off-topic classifier labels the natural language input as in-domain and then selecting a subdomain node in the hierarchy until an in-domain classifier labels the natural language input with a classification label, each of the plurality of domain nodes comprising a respective off-topic classifier and a respective in-domain classifier trained for a respective domain node;
training, by the computing device, the respective off-topic classifier for each domain node by applying positive training examples for a plurality of first samples from a respective domain and by providing negative training example for a plurality of second samples not from the respective domain; and
outputting, by the computing device, the classification label determined by the leaf node.

2. The method according to claim 1 , wherein receiving, by a computing device, a natural language input from a user further comprises:
receiving, by the computing device, the natural language input from the user comprising one or more of a voice utterance and a text utterance.

3. The method according to claim 1 , further comprising:
training, by the computing device, the plurality of domain nodes of a classification service for receiving user inputs from a plurality of users, each of the plurality of domain nodes for each providing text classification for a separate subservice of a plurality of subservices each separately provided by a separate domain of a plurality of domains provided by the classification service.

4. The method according to claim 1 , further comprising:
starting, by the computing device, routing the natural language input from the active domain node of the plurality of domain nodes, wherein the active domain node comprises a previous leaf node determined from classifying a previous natural language input received prior to the natural language input.

5. The method according to claim 1 , further comprising:
in response to receiving a natural language input from a user, determining, by the computing device, whether the natural language inputs is labeled as the classification label by a shared classifier that is shared among the plurality of domains, the shared classifier positively trained with a selection of sample utterances shared across the plurality of domains and negatively trained with a plurality of sampled in-domain utterances trained in the plurality of in-domain classifiers across the plurality of domains; and
in response to determining the natural language input is labeled as the classification label by the shared classifier, maintaining, by the computing device, the active node as the leaf node.

6. The method according to claim 1 , further comprising:
training, by the computing device, the respective in-domain classifier for each domain node by applying the positive training examples for a plurality of samples from a respective domain, wherein for a first selection of samples of the plurality of samples specific to the respective domain each of the first selection of samples is labeled with a respective classification label, wherein for a second selection of samples of the plurality of samples for each subdomain node each of the second selection of samples is labeled with an identifier for a respective subdomain node.

7. A computer system comprising one or more processors, one or more computer-readable memories, one or more computer-readable storage devices, and program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the stored program instructions comprising:
program instructions to receive a natural language input from a user;
program instructions to route the natural language input from an active domain node of a plurality of domain nodes of a multi-domain context-based hierarchy to a leaf node of the plurality of domain nodes by selecting a parent domain node in the hierarchy until an off-topic classifier labels the natural language input as in-domain and then selecting a subdomain node in the hierarchy until an in-domain classifier labels the natural language input with a classification label, each of the plurality of domain nodes comprising a respective off-topic classifier and a respective in-domain classifier trained for a respective domain node;
program instructions to train the respective off-topic classifier for each domain node by applying positive training examples for a plurality of first samples from a respective domain and by providing negative training example for a plurality of second samples not from the respective domain; and
program instructions to output the classification label determined by the leaf node.

8. The computer system according to claim 7 , the program instructions further comprising:
program instructions to receive the natural language input from the user comprising one or more of a voice utterance and a text utterance.

9. The computer system according to claim 7 , the program instructions further comprising:
program instructions to train the plurality of domain nodes of a classification service for receiving user inputs from a plurality of users, each of the plurality of domain nodes for each providing text classification for a separate subservice of a plurality of subservices each separately provided by a separate domain of a plurality of domains provided by the classification service.

10. The computer system according to claim 7 , the program instructions further comprising:
program instructions to start routing the natural language input from the active domain node of the plurality of domain nodes, wherein the active domain node comprises a previous leaf node determined from classifying a previous natural language input received prior to the natural language input.

11. The computer system according to claim 7 , the program instructions further comprising:
program instructions, in response to receiving a natural language input from a user, to determine whether the natural language inputs is labeled as the classification label by a shared classifier that is shared among the plurality of domains, the shared classifier positively trained with a selection of sample utterances shared across the plurality of domains and negatively trained with a plurality of sampled in-domain utterances trained in the plurality of in-domain classifiers across the plurality of domains; and
program instructions, in response to determining the natural language input is labeled as the classification label by the shared classifier, to maintain the active node as the leaf node.

12. The computer system according to claim 7 , the program instructions further comprising:
program instructions to train the respective in-domain classifier for each domain node by applying the positive training examples for a plurality of samples from a respective domain, wherein for a first selection of samples of the plurality of samples specific to the respective domain each of the first selection of samples is labeled with a respective classification label, wherein for a second selection of samples of the plurality of samples for each subdomain node each of the second selection of samples is labeled with an identifier for a respective subdomain node.

13. A computer

Assignees

• IBM

Inventors

• Tan Ming
• Kunc Ladislav
• Yu Yang
• Wang Haoyu
• Potdar Saloni

Classifications

• G06F16/353

  into predefined classes · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G10L15/1815

  Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning · CPC title

• G06N20/00

  Machine learning · CPC title