Multi-turn dialogue response generation with persona modeling

Patent metadata

Field	Value
Publication number	US-2021027770-A1
Application number	US-202016935784-A
Country	US
Kind code	A1
Filing date	Jul 22, 2020
Priority date	Jul 22, 2019
Publication date	Jan 28, 2021
Grant date	—

Abstract

Machine classifiers in accordance with embodiments of the invention capture long-term temporal dependencies in particular tasks, such as turn-based dialogues. Machine classifiers may be used to help users to perform tasks indicated by the user. When a user utterance is received, natural language processing techniques may be used to understand the user's intent. Templates may be determined based on the user's intent in the generation of responses to solicit information from the user. A variety of persona attributes may be determined for a user. The persona attributes may be determined based on the user's utterances and/or provided as metadata included with the user's utterances. A response persona may be used to generate responses to the user's utterances such that the generated responses match a tone appropriate to the task. A response persona may be used to generate templates to solicit additional information and/or generate responses appropriate to the task.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
initializing a machine classifier having a sequence to sequence network architecture, wherein the sequence to sequence network architecture comprises an encoder and a decoder; obtaining input data comprising a conversation history and a user utterance of a user; determining a user intent based on the user utterance; determining, by the machine classifier and based on the conversation history and the user utterance, a user persona for the user; generating, by the machine classifier and based on the user intent and the user persona, a response persona; generating, by the machine classifier and based on the user utterance and the response persona, a response; and providing the response.

2 . The computer-implemented method of claim 1 , wherein the response persona and the response are generated in parallel.

3 . The computer-implemented method of claim 1 , further comprising training the machine classifier based on a training set comprising a plurality of encoder sequences and a plurality of decoder sequences by:
generating an encoding of each encoder sequence and each decoder sequence in the training set; selecting a subset of the encodings; appending an informative padding to each of the selected subset of encodings; prepending a start of sequence token to each of the encodings of the encoder sequences; appending an end of sequence token to each of the encodings of the decoder sequences; and for each encoding of the encoder sequences:
training the encoder using the encoding of the encoder sequence; and
training the decoder using the encoding of the decoder sequence corresponding to the encoder sequence.

4 . The computer-implemented method of claim 1 , wherein generating the response comprises:
generating an input encoding of the input data; generating an output sequence comprising a start of sequence token; completing the output sequence by:
generating a next output sequence token by providing the input encoding to the machine classifier;
appending the next output sequence token to the output sequence; and
iteratively generating next output sequence tokens by providing the input encoding to the machine classifier and appending each generated next output sequence token to the output sequence until the generated subsequent next output sequence token comprises an end of sequence token; and
generating the response based on the output sequence.

5 . The computer-implemented method of claim 1 , further comprising:
obtaining, by the machine classifier and based on the user intent, a response template; and generating the response further based on the response template.

6 . The computer-implemented method of claim 5 , wherein the response template is generated based on the user persona.

7 . The computer-implemented method of claim 5 , wherein the response template is generated based on the response persona.

8 . The computer-implemented method of claim 1 , wherein the user persona comprises attributes selected from the group consisting of a speaker's identity, a speaker's background, a speaker's location, and a speaker's preference.

9 . The computer-implemented method of claim 1 , wherein:
the input data comprises a multi-turn dialog indicating a class of task; and the method further comprises generating the response persona based on the class of task.

10 . The computer-implemented method of claim 9 , further comprising generating the user persona based on the class of task.

11 . A device, comprising:
a processor; and a memory in communication with the processor and storing instructions that, when read by the processor, cause the device to:
initialize a machine classifier having a sequence to sequence network architecture, wherein the sequence to sequence network architecture comprises an encoder and a decoder;
obtain input data comprising a conversation history and a user utterance of a user;
determine a user intent based on the user utterance;
determine a class of task based on the conversation history and the user utterance;
determine, by the machine classifier and based on the conversation history, the user utterance, and the class of task, a user persona for the user;
generate, by the machine classifier and based on the user intent, the user persona, and the class of task, a response persona;
generate, by the machine classifier and based on the user utterance and the response persona, a response; and
provide the response.

12 . The device of claim 11 , wherein the response persona and the response are generated in parallel.

13 . The device of claim 11 , wherein the instructions, when read by the processor, further cause the device to train the machine classifier based on a training set comprising a plurality of encoder sequences and a plurality of decoder sequences by causing the device to:
generate an encoding of each encoder sequence and each decoder sequence in the training set; select a subset of the encodings; append an informative padding to each of the selected subset of encodings; prepend a start of sequence token to each of the encodings of the encoder sequences; append an end of sequence token to each of the encodings of the decoder sequences; and for each encoding of the encoder sequences:
train the encoder using the encoding of the encoder sequence; and
train the decoder using the encoding of the decoder sequence corresponding to the encoder sequence.

14 . The device of claim 11 , wherein the instructions, when read by the processor, further cause the device to generate the response by causing the device to:
generate an input encoding of the input data; generate an output sequence comprising a start of sequence token; complete the output sequence by:
generating a next output sequence token by providing the input encoding to the machine classifier;
appending the next output sequence token to the output sequence; and
iteratively generating next output sequence tokens by providing the input encoding to the machine classifier and appending each generated next output sequence token to the output sequence until the generated subsequent next output sequence token comprises an end of sequence token; and
generate the response based on the output sequence.

15 . The device of claim 11 , wherein the instructions, when read by the processor, further cause the device to:
obtain, by the machine classifier and based on the user intent, a response template; and generate the response further based on the response template.

16 . The device of claim 15 , wherein the response template is generated based on the user persona.

17 . The device of claim 15 , wherein the response template is generated based on the response persona.

18 . A non-transitory machine-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform steps comprising:
initializing a machine classifier having a sequence to sequence network architecture, wherein the sequence to sequence network architecture comprises an encoder and a decoder; training the machine classifier based on a training set comprising a plurality of encoder sequences and a plurality of decoder sequences, wherein training the machine classifier comprises:
generating an encoding of each encoder sequence and each decoder sequence in the training set;
selecting a subset of the encodings;
appending an informative padding to each of the selected subset of encodings;
prepending a s

Assignees

• Capital One Services Llc

Inventors

• Olabiyi Oluwatobi
• Mueller Erik T
• Zhang Rui
• Kulis Zachary
• Singh Varun

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F40/56

  Natural language generation · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title