Multi-turn dialogue response generation using asymmetric adversarial machine classifiers

Patent metadata

Field	Value
Publication number	US-11836452-B2
Application number	US-202318114567-A
Country	US
Kind code	B2
Filing date	Feb 27, 2023
Priority date	Aug 19, 2020
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

In a variety of embodiments, machine classifiers may model multi-turn dialogue as a one-to-many prediction task. The machine classifier may be trained using adversarial bootstrapping between a generator and a discriminator with multi-turn capabilities. The machine classifiers may be trained in both auto-regressive and traditional teacher-forcing modes, with the generator including a hierarchical recurrent encoder-decoder network and the discriminator including a bi-directional recurrent neural network. The discriminator input may include a mixture of ground truth labels, the teacher-forcing outputs of the generator, and/or noise data. This mixture of input data may allow for richer feedback on the autoregressive outputs of the generator. The outputs can be ranked based on the discriminator feedback and a response selected from the ranked outputs.

First claim

What is claimed is:

1. A computer-implemented method for generating a response, comprising:
receiving communication data comprising a plurality of dialog sequences;
generating, by a generator of a generative adversarial network and based on context embedding and word embedding of the communication data, a plurality of responses, wherein each response in the plurality of responses comprises at least one keyword selected based on a maximum likelihood estimation of the at least one keyword in the communication data;
ranking, by at least one discriminator of the generative adversarial network and based on the context embedding and the word embedding, the plurality of responses;
selecting, by the at least one discriminator, an optimal response from among the ranked plurality of responses; and
transmitting the optimal response.

2. The computer-implemented method of claim 1 , wherein the generator and the at least one discriminator share the context embedding and the word embedding.

3. The computer-implemented method of claim 1 , further comprising:
generating, by an encoder and based on the communication data, the word embedding of the communication data.

4. The computer-implemented method of claim 1 , further comprising:
generating, by the generator and based on training data comprising conversation data, initial autoregression data;
generating, by the generator and based on a set of teacher forcing samples, initial teacher forcing data; and
determining, by the at least one discriminator and based on the initial autoregression data and the initial teacher forcing data, a discriminator accuracy.

5. The computer-implemented method of claim 1 , further comprising:
training the at least one discriminator after determining that a discriminator accuracy is below a discriminator threshold value, wherein the discriminator accuracy is determined by the at least one discriminator and based on an initial autoregression data and an initial teacher forcing data;
retraining the generator using a teacher forcing loss function of the generator after determining that the discriminator accuracy is below a generator threshold value;
retraining the generator using the teacher forcing loss function and an autoregressive loss function after determining that the discriminator accuracy is above the generator threshold value; and
storing a trained generative adversarial network.

6. The computer-implemented method of claim 1 , further comprising:
determining, by the at least one discriminator, a discriminator score for each response in the plurality of responses indicating a statistical similarity between the response and an anticipated ground truth response to the communication data; and
ranking the plurality of the responses further based on the discriminator score for each response in the plurality of responses.

7. The computer-implemented method of claim 1 , further comprising:
determining, by the at least one discriminator, a discriminator score for each response in the plurality of responses indicating a probability that the response is a ground truth response to a current prompt indicated in the communication data; and
ranking the plurality of the responses further based on the discriminator score for each response in the plurality of responses.

8. The computer-implemented method of claim 1 , wherein the generator comprises hierarchical recurrent encoder-decoder network.

9. The computer-implemented method of claim 1 , wherein the at least one discriminator comprises a convolutional neural network.

10. The computer-implemented method of claim 1 , wherein the at least one discriminator comprises a recurrent neural network.

11. A computing device for generating responses, comprising:
a processor; and
a memory in communication with the processor and storing instructions that, when executed by the processor, cause the computing device to:
receive communication data comprising a plurality of dialog sequences;
generate, by the computing device comprising a generator of a generative adversarial network and based on context embedding and word embedding of the communication data, a plurality of responses, wherein each response in the plurality of responses comprises at least one keyword selected based on a maximum likelihood estimation of the at least one keyword in the communication data;
rank, by the computing device comprising at least one discriminator of the generative adversarial network and based on the context embedding and the word embedding, the plurality of responses;
select, by the at least one discriminator, an optimal response from among the ranked plurality of responses; and
transmit the optimal response.

12. The computing device of claim 11 , wherein the generator and the at least one discriminator share the context embedding and the word embedding.

13. The computing device of claim 11 , wherein the instructions, when read by the processor, further cause the computing device to:
generate, by an encoder and based on the communication data, the word embedding of the communication data.

14. The computing device of claim 11 , wherein the instructions, when read by the processor, further cause the computing device to:
generate, by the generator and based on training data comprising conversation data, initial autoregression data;
generate, by the generator and based on a set of teacher forcing samples, initial teacher forcing data; and
determine, by the at least one discriminator and based on the initial autoregression data and the initial teacher forcing data, a discriminator accuracy.

15. The computing device of claim 11 , wherein the instructions, when read by the processor, further cause the computing device to:
train the at least one discriminator after determining that a discriminator accuracy is below a discriminator threshold value, wherein the discriminator accuracy is determined by the at least one discriminator and based on an initial autoregression data and an initial teacher forcing data;
retrain the generator using a teacher forcing loss function of the generator after determining that the discriminator accuracy is below a generator threshold value;
retrain the generator using the teacher forcing loss function and an autoregressive loss function after determining that the discriminator accuracy is above the generator threshold value; and
store a trained generative adversarial network.

16. The computing device of claim 11 , wherein the instructions, when read by the processor, further cause the computing device to:
determine, by the at least one discriminator, a discriminator score for each response in the plurality of responses indicating a statistical similarity between the response and an anticipated ground truth response to the communication data; and
rank the plurality of the responses further based on the discriminator score for each response in the plurality of responses.

17. The computing device of claim 11 , wherein the instructions, when read by the processor, further cause the computing device to:
determine, by the at least one discriminator, a discriminator score for each response in the plurality of responses indicating a probability that the response is a ground truth response to a current prompt indicated in the communication data; and
rank the plurality of the responses further based on the discriminator score for each response in the plurality of responses.

18. A non-transitory machine-readable medium storing instructions that, when executed by one or more processors, cause the one or more process

Assignees

• Capital One Services Llc

Inventors

• Olabiyi Oluwatobi
• Mueller Erik T

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title