Processing an incomplete message with a neural network to generate suggested messages

Patent metadata

Field	Value
Publication number	US-11216510-B2
Application number	US-201816054473-A
Country	US
Kind code	B2
Filing date	Aug 3, 2018
Priority date	Aug 3, 2018
Publication date	Jan 4, 2022
Grant date	Jan 4, 2022

Abstract

Text of an incomplete message entered by a user may be processed using a neural network to suggest messages similar to the message the user is in the process of entering. Word embeddings may be obtained for the words of the text that represent the words in a first vector space. The word embeddings may then be processed by the neural network to compute an input message feature vector that represents the incomplete message in a second vector space. The input message feature vector may be used to select a first designated message as a suggestion from a set of designated messages, and the first designated message may be selected using a similarity score computed from the input message feature vector and a first designated message feature vector corresponding to the first designated message. The first designated message may then be presented as a suggestion to the user.

First claim

What is claimed is:

1. A computer-implemented method for suggesting a message using a neural network, the method comprising:
receiving text of an incomplete message as the incomplete message is being entered by a user;
obtaining word embeddings for the text of the incomplete message, wherein a word embedding represents a corresponding word in a first vector space;
processing the word embeddings with the neural network to compute an input message feature vector, wherein the input message feature vector represents the incomplete message in a second vector space;
obtaining a set of designated messages, wherein each designated message is associated with a designated message feature vector, and a designated message feature vector represents a corresponding designated message in the second vector space;
computing a similarity score for each designated message of the set of designated messages by processing the input message feature vector and the designated message feature vectors, wherein a first similarity score for a first designated message is computed using the input message feature vector and a first designated message feature vector corresponding to the first designated message;
selecting the first designated message from the set of designated messages using the first similarity score;
presenting the first designated message to the user;
receiving a selection of the first designated message from the user;
determining a response by processing the selection of the first designated message; and
transmitting the response to the user.

2. The computer-implemented method of claim 1 , wherein the user is a customer of a company and the text of the incomplete message relates to a customer support request.

3. The computer-implemented method of claim 1 , comprising:
selecting a plurality of designated messages from the set of designated messages, wherein each of the plurality of designated messages is a prefix match for the text of the incomplete message;
computing a prefix score for each of the plurality of designated messages; and
selecting the first designated message using the prefix scores.

4. The computer-implemented method of claim 3 , wherein the prefix score indicates a frequency of the corresponding designated message in a corpus of messages.

5. The computer-implemented method of claim 3 , wherein:
the plurality of designated messages comprises the first designated message;
selecting the first designated message comprises using a first prefix score corresponding to the first designated message and the first similarity score.

6. The computer-implemented method of claim 5 , comprising:
computing a first selection score for the incomplete message using the first prefix score and the first similarity score; and
selecting the first designated message using the first selection score.

7. The computer-implemented method of claim 6 , wherein the first selection score is a maximum of the first prefix score and the first similarity score.

8. The computer-implemented method of claim 1 , comprising:
receiving second text of a second incomplete message as the second incomplete message is being entered by the user;
computing a second input message feature vector using the second text and the neural network; and
deciding not to suggest any messages to the user.

9. A system for suggesting a message, the system comprising:
at least one server computer comprising at least one processor and at least one memory, the at least one server computer configured to:
receive text of an incomplete message as the incomplete message is being entered by a user;
obtain word embeddings for the text of the incomplete message, wherein a word embedding represents a corresponding word in a first vector space;
process the word embeddings with a neural network to compute an input message feature vector, wherein the input message feature vector represents the incomplete message in a second vector space;
obtain a set of designated messages, wherein each designated message is associated with a designated message feature vector, and a designated message feature vector represents a corresponding designated message in the second vector space;
compute a similarity score for each designated message of the set of designated messages by processing the input message feature vector and the designated message feature vectors, wherein a first similarity score for a first designated message is computed using the input message feature vector and a first designated message feature vector corresponding to the first designated message;
select the first designated message from the set of designated messages using the first similarity score; and
present the first designated message to the user.

10. The system of claim 9 , wherein the user is a customer of a company and the system is implemented by a third-party company providing services to the company.

11. The system of claim 9 , wherein the neural network was trained using a denoising auto-encoder.

12. The system of claim 11 , wherein the neural network was trained by:
obtaining a training message from a training corpus;
generating a noisy training message by modifying the training message;
setting an input of the denoising auto-encoder to the noisy training message;
setting an output of the denoising auto-encoder to the training message; and
updating parameters of the denoising auto-encoder using stochastic gradient descent.

13. The system of claim 9 , wherein the at least one server computer is further configured to receive an edit by the user of the first designated message.

14. The system of claim 9 , wherein the at least one server computer is configured to:
select a second designated message from the set of designated messages using a second similarity score corresponding to the second designated message; and
present the second designated message to the user.

15. The system of claim 9 , wherein selecting the first designated message comprises:
obtaining a first probability score for the first designated message;
computing a first selection score for the first designated message using the first probability score and the first similarity score; and
selecting the first designated message using the first selection score.

16. The system of claim 15 , wherein the first probability score indicates a frequency of the first designated message in a corpus of messages.

17. One or more non-transitory, computer-readable media comprising computer-executable instructions that, when executed, cause at least one processor to perform actions comprising:
receiving text of an incomplete message as the incomplete message is being entered by a user;
obtaining word embeddings for the text of the incomplete message, wherein a word embedding represents a corresponding word in a first vector space;
processing the word embeddings with a neural network to compute an input message feature vector, wherein the input message feature vector represents the incomplete message in a second vector space;
obtaining a set of designated messages, wherein each designated message is associated with a designated message feature vector, and a designated message feature vector represents a corresponding designated message in the second vector space;
computing a similarity score for each designated message of the set of designated messages by processing the input message feature vector and the designated message feature vectors, wherein a first similarity score for a first designated message is computed using the input message feature vector and a first designate

Assignees

• Asapp Inc

Inventors

• Jiang Lisa Lijia
• Lei Tao
• Henry Shawn

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title