Method and system for processing sentence, and electronic device

Patent metadata

Field	Value
Publication number	US-12039281-B2
Application number	US-201917263836-A
Country	US
Kind code	B2
Filing date	Jul 18, 2019
Priority date	Jul 27, 2018
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

The present disclosure provides a method for processing sentence, including: determining a vector representation of a sentence to be processed; inputting the vector representation of the sentence to be processed to a first recurrent neural network, and updating a state of a hidden layer of the first recurrent neural network as a vector representation of a historical dialogue, wherein, if a processed sentence exists, the state of the hidden layer of the first recurrent neural network is related to the processed sentence; determining a knowledge vector related to the sentence to be processed; and generating a reply sentence based on the vector representation of the historical dialogue and the knowledge vector. The present disclosure further provides a system for processing sentence, an electronic device, and a computer-readable medium.

First claim

What is claimed is:

1. A method for processing sentence, comprising:
determining a vector representation of a sentence to be processed;
inputting the vector representation of the sentence to be processed to a first recurrent neural network, and updating a state of a hidden layer of the first recurrent neural network as a vector representation of a historical dialogue, wherein, if a vector representation of another sentence is input to the first recurrent neural network to obtain a processed sentence before inputting the vector representation of the sentence to be processed to the first recurrent neural network-exists, the state of the hidden layer of the first recurrent neural network is related to the processed sentence;
determining a knowledge vector related to the sentence to be processed; and
generating a reply sentence based on the vector representation of the historical dialogue and the knowledge vector.

2. The method according to claim 1 , wherein the determining a vector representation of the sentence to be processed comprises:
converting words in the sentence to be processed into word vectors; and
inputting the word vectors to a second recurrent neural network in order, and obtaining a state of a hidden layer of the second recurrent neural network as a vector representation of the sentence to be processed, wherein the state of the hidden layer of the second recurrent neural network is unrelated to the processed sentence.

3. The method according to claim 1 , wherein the determining a knowledge vector related to the sentence to be processed comprises:
determining a plurality of candidate entries related to the sentence to be processed;
determining candidate entry vectors of the candidate entries;
determining a correlation coefficient between each candidate entry vector and the vector representation of the sentence to be processed, or determining a correlation coefficient between each candidate entry vector and the vector representation of the historical dialogue; and
determining the knowledge vector based on the correlation coefficients and the candidate entry vectors.

4. The method according to claim 3 , wherein the determining a plurality of candidate entries related to the sentence to be processed comprises at least one of the following:
determining candidate entries that match the sentence to be processed;
determining candidate entries that are similar to entity content in the sentence to be processed; or
if a processed sentence exists, determining candidate entries related to the processed sentence.

5. The method according to claim 3 , further comprising:
ignoring a part of the candidate entries in a case that a number of the candidate entries exceeds a preset number.

6. The method according to claim 1 , wherein the generating a reply sentence based on the vector representation of the historical dialogue and the knowledge vector comprises:
inputting the vector representation of the historical dialogue, the knowledge vector, and a word vector of a previously predicted word to a third recurrent neural network, determining a currently predicted word based on a state of a hidden layer of the third recurrent neural network, and performing above operations repeatedly until a complete reply sentence is generated, wherein:
the state of the hidden layer of the third recurrent neural network is unrelated to the processed sentence; and
if no previously predicted word exists, a default start symbol is used as the previously predicted word.

7. An electronic device comprising:
one or more processors; and
a memory configured to store one or more computer programs, which when executed by one or more processors, cause the one or more processors to implement the method according to claim 1 .

8. A non-transitory computer-readable medium having executable instructions stored thereon, which when executed by a processor, cause the processor to implement the method according to claim 1 .

9. An electronic device comprising:
one or more processors; and
a memory configured to store one or more computer programs, which when executed by one or more processors, cause the one or more processors to implement the method according to claim 2 .

10. A non-transitory computer-readable medium having executable instructions stored thereon, which when executed by a processor, cause the processor to implement the method according to claim 2 .

11. An electronic device comprising:
one or more processors; and
a memory configured to store one or more computer programs, which when executed by one or more processors, cause the one or more processors to implement the method according to claim 3 .

12. A non-transitory computer-readable medium having executable instructions stored thereon, which when executed by a processor, cause the processor to implement the method according to claim 3 .

13. An electronic device comprising:
one or more processors; and
a memory configured to store one or more computer programs, which when executed by one or more processors, cause the one or more processors to implement the method according to claim 4 .

14. A non-transitory computer-readable medium having executable instructions stored thereon, which when executed by a processor, cause the processor to implement the method according to claim 4 .

15. A system for processing sentence, comprising:
a sentence to be processed vector determination module configured to determine a vector representation of a sentence to be processed;
a historical dialogue vector determination module configured to input the vector representation of the sentence to be processed to a first recurrent neural network, and update a state of a hidden layer of the first recurrent neural network as a vector representation of a historical dialogue, wherein, if a vector representation of another sentence is input to the first recurrent neural network to obtain a processed sentence before inputting the vector representation of the sentence to be processed to the first recurrent neural network, the state of the hidden layer of the first recurrent neural network is related to the processed sentence;
a knowledge vector determination module configured to determine a knowledge vector related to the sentence to be processed; and
a reply sentence generation module configured to generate a reply sentence based on the vector representation of the historical dialogue and the knowledge vector.

16. The system according to claim 15 , wherein the sentence to be processed vector determination module comprises:
a word vector conversion sub-module configured to convert words in the sentence to be processed into word vectors; and
a sentence to be processed vector determination sub-module configured to input the word vectors into a second recurrent neural network in order, and obtain a state of a hidden layer of the second recurrent neural network as a vector representation of the sentence to be processed, wherein the state of the hidden layer of the second recurrent neural network is unrelated to the processed sentence.

17. The system according to claim 15 , wherein the knowledge vector determination module comprises:
a candidate entry determination sub-module configured to determine a plurality of candidate entries related to the sentence to be processed;
a candidate entry vector determination sub-module configured to determine candidate entry vectors of the candidate entries;
a correlation coefficient determination sub-module configured to determine a correlation coefficient between each candidate entry vector and the vector repr

Assignees

• Beijing Jingdong Shangke Information Technology Co Ltd
• Beijing Jingdong Century Trading Co Ltd

Inventors

• Chen Hongshen
• Liu Shuman
• Ren Zhaochun
• Yin Dawei
• Zhao Yihong

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06F40/56

  Natural language generation · CPC title