Customized message suggestion with user embedding vectors

Patent metadata

Field	Value
Publication number	US-12470503-B2
Application number	US-202217856752-A
Country	US
Kind code	B2
Filing date	Jul 1, 2022
Priority date	Oct 25, 2019
Publication date	Nov 11, 2025
Grant date	Nov 11, 2025

Abstract

A message may be suggested to a user participating in a conversation using one or more neural networks where the suggested message is adapted to the preferences or communication style of the user. The suggested message may be adapted to the user with a user embedding vector that represents the preferences or communication style of the user in a vector space. To suggest a message to the user, a conversation feature vector may be computed by processing the text of the conversation with a neural network. A context score may be computed for one or more designated messages, where the context score is computed by processing the user embedding vector, the conversation feature vector, and a designated message feature vector with a neural network. A designated message may be selected as a suggested message for the user using the context scores. The suggestion may then be presented to the user.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
obtaining a user embedding vector corresponding to a user, wherein the user embedding vector represents the user in a first vector space; receiving text of a conversation with the user; computing a conversation feature vector using the text of the conversation and a first neural network, wherein the conversation feature vector represents the conversation in a second vector space; obtaining a set of designated messages, wherein each designated message is associated with a corresponding designated message feature vector; computing a first context score for a first designated message of the set of designated messages by processing the user embedding vector, the conversation feature vector, and a first designated message feature vector with a second neural network; selecting the first designated message using the first context score; and presenting the first designated message as a suggested message to the user.

2 . The computer-implemented method of claim 1 , wherein computing the conversation feature vector comprises:
obtaining word embeddings for the text of the conversation, wherein a word embedding represents a corresponding word in a third vector space; and processing the word embeddings with the first neural network.

3 . The computer-implemented method of claim 1 , comprising:
selecting a second designated message using the first context score; and presenting the second designated message as a suggested message to the user.

4 . The computer-implemented method of claim 1 , comprising:
obtaining text encodings for text of the first designated message; and computing the first designated message feature vector by processing the text encodings for the text of the first designated message with a response encoder neural network.

5 . The computer-implemented method of claim 1 , comprising:
obtaining input text entered by the user; computing a first input text score for the first designated message; and selecting the first designated message using the first input text score.

6 . The computer-implemented method of claim 1 , comprising:
obtaining input text entered by the user; obtaining text encodings for the input text; computing an input text feature vector by processing the text encodings of the input text with a response encoder neural network; computing a first semantic score for the first designated message using the input text feature vector and the first designated message feature vector; and selecting the first designated message using the first semantic score.

7 . The computer-implemented method of claim 6 , wherein computing the first semantic score comprises computing a distance or a cosine similarity between the input text feature vector and the first designated message feature vector.

8 . The computer-implemented method of claim 1 , wherein the second neural network comprises a multi-layer perceptron.

9 . A system, comprising:
at least one server computer comprising at least one processor and at least one memory, the at least one server computer configured to:
obtain a user embedding vector corresponding to a user, wherein the user embedding vector represents the user in a first vector space;
receive text of a conversation with the user;
compute a conversation feature vector using the text of the conversation and a first neural network, wherein the conversation feature vector represents the conversation in a second vector space;
obtain a set of designated messages, wherein each designated message is associated with a corresponding designated message feature vector;
compute a first context score for a first designated message of the set of designated messages by processing the user embedding vector, the conversation feature vector, and a first designated message feature vector with a second neural network;
select the first designated message using the first context score; and
present the first designated message as a suggested message to the user.

10 . The system of claim 9 , wherein the user embedding vector is trained using messages sent by the user and represents a conversational style or conversational preferences of the user.

11 . The system of claim 9 , wherein the user is a customer support agent of a company and the system is implemented by a third-party company providing services to the company.

12 . The system of claim 9 , wherein the at least one server computer is configured to:
obtain input text entered by the user; obtain a first prefix score for the first designated message by comparing the input text to a prefix of the first designated message; and select the first designated message using the first prefix score.

13 . The system of claim 12 , wherein a prefix score has a first value if the input text is not a prefix of a corresponding designated message and has a second value if the input text is a prefix of the corresponding designated message.

14 . The system of claim 9 , wherein the at least one server computer is configured to:
compute a first semantic score for the first designated message; compute a first prefix score for the first designated message; and select the first designated message using the first context score and the first prefix score.

15 . The system of claim 9 , wherein the first neural network comprises a recurrent neural network.

16 . The system of claim 9 , wherein the at least one server computer is configured to compute the first context score by processing conversation metadata with the second neural network.

17 . One or more non-transitory, computer-readable media comprising computer-executable instructions that, when executed, cause at least one processor to perform actions comprising:
obtaining a user embedding vector corresponding to a user, wherein the user embedding vector represents the user in a first vector space; receiving text of a conversation with the user; computing a conversation feature vector using the text of the conversation and a first neural network, wherein the conversation feature vector represents the conversation in a second vector space; obtaining a set of designated messages, wherein each designated message is associated with a corresponding designated message feature vector; computing a first context score for a first designated message of the set of designated messages by processing the user embedding vector, the conversation feature vector, and a first designated message feature vector with a second neural network; selecting the first designated message using the first context score; and presenting the first designated message as a suggested message to the user.

18 . The one or more non-transitory, computer-readable media of claim 17 , wherein the conversation is between the user and an automated natural language processing system.

19 . The one or more non-transitory, computer-readable media of claim 17 , wherein the second neural network was trained using negative sampling.

20 . The one or more non-transitory, computer-readable media of claim 17 , wherein the user embedding vector was computed during training of the second neural network.

21 . The one or more non-transitory, computer-readable media of claim 17 , wherein the set of designated messages was obtained from a data store of customer support messages of a company.

Assignees

• Asapp Inc

Inventors

• Ball Kelsey Taylor
• Lei Tao
• Fox Christopher David
• Hackman Joseph Ellsworth

Classifications

• G06F40/56 Primary

  Natural language generation · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title