Out of domain sentence detection

Patent metadata

Field	Value
Publication number	US-12443796-B2
Application number	US-202217815630-A
Country	US
Kind code	B2
Filing date	Jul 28, 2022
Priority date	Jul 28, 2022
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

A method, a structure, and a computer system for OOD sentence detection in dialogue systems. The exemplary embodiments may include receiving, for a domain corresponding to a particular topic, one or more on-topic text inputs and one or more off-topic text inputs. The exemplary embodiments may further include encoding the one or more on-topic text inputs and the one or more off-topic text inputs into a latent space, as well as decoding the one or more on-topic text inputs and the one or more off-topic text inputs from the latent space. The exemplary embodiments may additionally include minimizing a reconstruction error between the encoded one or more on-topic text inputs and the decoded one or more on-topic text inputs, and maximizing a reconstruction error between the encoded one or more off-topic text inputs and the decoded one or more off-topic text inputs.

First claim

The invention claimed is:

1. A computer-implemented method, comprising:
training, by at least one processor of a device, an autoencoder to detect user inputs that are out of a domain in which a dialogue system has been trained, wherein the training comprises:
receiving, for the domain of the dialogue system, training user input representations, wherein the training user input representations comprise respective weights based on respective importance of the training user input representations to the training, and wherein the training user input representations comprise in-domain training user input representations and out-of-domain training user input representations;
encoding the in-domain training user input representations into encoded in-domain training user input representations and the out-of-domain training user input representations into encoded out-of-domain training user input representations in a latent space;
decoding, from the latent space, the encoded in-domain training user input representations into decoded in-domain training user input representations and the encoded out-of-domain training user input representations into decoded out-of-domain training user input representations;
minimizing an in-domain reconstruction error between the in-domain training user input representations and the decoded in-domain training user input representations respectively corresponding to the in-domain training user input representations, wherein the in-domain reconstruction error is based on the respective weights of the in-domain training user input representations; and
maximizing an out-of-domain reconstruction error between the out-of-domain training user input representations and the decoded out-of-domain training user input representations respectively corresponding to the out-of-domain training user input representations, wherein the out-of-domain reconstruction error is based on the respective weights of the out-of-domain training user input representations;

receiving, by the at least one processor, using the dialogue system, a user input from a user;
determining, by the at least one processor, using the autoencoder, whether the user input is out of the domain; and
responsive to determining that the user input is out of the domain, directing the user to a different dialogue system trained for another domain that is different from the domain.

2. The computer-implemented method of claim 1 , wherein the encoding the in-domain training user input representations and the out-of-domain training user input representations into the latent space comprises:
converting, using at least one sentence encoder, the in-domain training user input representations into respective in-domain vectors;
converting, using the at least one sentence encoder, the out-of-domain training user input representations into respective out-of-domain vectors;
encoding, using the autoencoder, the respective in-domain vectors into respective in-domain latent inputs within the latent space; and
encoding, using the autoencoder, the respective out-of-domain vectors into one or more on topic latent inputs and one or more off topic respective out-of-domain latent inputs within the latent space.

3. The computer-implemented method of claim 2 , wherein the decoding the encoded in-domain training user input representations and the encoded out-of-domain training user input representations from the latent space comprises:
decoding, using the autoencoder, the respective in-domain latent inputs into the decoded in-domain training user input representations; and
decoding, using the autoencoder, the respective out-of-domain latent inputs into the decoded out-of-domain training user input representations.

4. The computer-implemented method of claim 1 , wherein the minimizing the in-domain reconstruction error comprises:
assigning a positive factor for a loss function between the encoded in-domain training user input representations and the decoded in-domain training user input representations.

5. The computer-implemented method of claim 1 , wherein the maximizing the out-of-domain reconstruction error comprises:
assigning a negative factor for a loss function between the encoded out-of-domain training user input representations and the decoded out-of-domain training user input representations.

6. The computer-implemented method of claim 1 , further comprising:
utilizing the in-domain reconstruction error and the out-of-domain reconstruction error as an indicator as to whether a user dialogue comprising the user input is in-domain or out-of-domain.

7. The computer-implemented method of claim 1 , wherein the training user input representations comprise previous user-provided inputs of the dialogue system.

8. A computer program product comprising a non-transitory computer-readable medium having instructions stored thereon that, in response to execution, cause a processor to perform operations comprising:
training an autoencoder to detect user inputs that are out of a domain in which a dialogue system has been trained, wherein the training comprises:
receiving, for the domain of the dialogue system, training user input representations, wherein the training user input representations comprise respective weights based on respective importance of the training user input representations to the training, and wherein the training user input representations comprise in-domain training user input representations and out-of-domain training user input representations;
encoding the in-domain training user input representations into encoded in-domain training user input representations and the out-of-domain training user input representations into encoded out-of-domain training user input representations in a latent space;
decoding, from the latent space, the encoded in-domain training user input representations into decoded in-domain training user input representations and the encoded out-of-domain training user input representations into decoded out-of-domain training user input representations;
minimizing an in-domain reconstruction error between the in-domain training user input representations and the decoded in-domain training user input representations respectively corresponding to the in-domain training user input representations, wherein the in-domain reconstruction error is based on the respective weights of the in-domain training user input representations; and
maximizing an out-of-domain reconstruction error between the out-of-domain training user input representations and the decoded out-of-domain training user input representations respectively corresponding to the out-of-domain training user input representations, wherein the out-of-domain reconstruction error is based on the respective weights of the out-of-domain training user input representations;

receiving, using the dialogue system, a user input from a user;
determining, using the autoencoder, whether the user input is out of the domain; and
responsive to determining that the user input is out of the domain, directing the user to a different dialogue system trained for another domain that is different from the domain.

9. The computer program product of claim 8 , wherein the encoding the in-domain training user input representations and the out-of-domain training user input representations into the latent space comprises:
converting, using at least one sentence encoder, the in-domain training user input representations into respective in-domain vectors;
converting, using the at least one sentence encoder, the out-of-domain training user input representations into respective out-of-domain vectors;
encoding, using the autoencoder, the respective in-domain vectors into respective in-domain latent inputs

Assignees

• IBM

Inventors

• Qi Haode
• Qian Cheng
• Kunc Ladislav
• Potdar Saloni
• Wayne Eric Donald

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F16/3329

  Natural language query formulation · CPC title

• G06F16/36

  Creation of semantic tools, e.g. ontology or thesauri · CPC title

• G06F40/279 Primary

  Recognition of textual entities · CPC title