Method for machine reading comprehension

Patent metadata

Field	Value
Publication number	US-11620449-B2
Application number	US-202017024726-A
Country	US
Kind code	B2
Filing date	Sep 18, 2020
Priority date	Sep 19, 2019
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

A method for machine reading comprehension includes: S 1 , obtaining a character-level indication vector of a question and a character-level indication vector of an article; S 2 , obtaining an encoded question vector and an encoded article vector; S 3 , obtaining an output P1 of a bidirectional attention model and an output P2 of a shared attention model; S 4 , obtaining an aggregated vector P3; S 5 , obtaining a text encoding vector P4; S 6 , obtaining global interaction information between words within the article; S 7 , obtaining a text vector P5 after using the self-attention model; S 8 , obtaining aggregated data P6 according to the text encoding vector P4 and the text vector P5; S 9 , obtaining a context vector of the article according to the aggregated data P6 and an unencoded article vector P; and S 10 , predicting an answer position according to the context vector of the article and the encoded question vector to complete the machine reading comprehension.

First claim

What is claimed is:

1. A method for machine reading comprehension, comprising the following steps:
S 0 : obtaining a question in words from a user and obtaining an article in words containing an answer to the question;
S 1 : encoding the question words and the article words, respectively, to obtain a character-level indication vector of the question and a character-level indication vector of the article, respectively;
S 2 : encoding the character-level indication vector of the question and the character-level indication vector of the article, respectively, to obtain an encoded question vector and an encoded article vector, respectively;
S 3 : inputting the encoded article vector into a bidirectional attention model to obtain an output P1 of the bidirectional attention model, and inputting the encoded question vector into a shared attention model to obtain an output P2 of the shared attention model;
S 4 : aggregating the output P1 of the bidirectional attention model and the output P2 of the shared attention model according to an aggregation mechanism to obtain an aggregated vector P3;
S 5 : aggregating the aggregated vector P3 and an unencoded article vector P to obtain a text encoding vector P4;
S 6 : obtaining global interaction information between words within the article based on the text encoding vector P4 according to a self-attention model;
S 7 : obtaining a text vector P5 after using the self-attention model according to the global interaction information and the text encoding vector P4;
S 8 : aggregating the text encoding vector P4 and the text vector P5 after using the self-attention model according to an aggregation function to obtain aggregated data P6;
S 9 : splicing the aggregated data P6 and the unencoded article vector P to obtain spliced data, sending the spliced data as an input into a bidirectional gated recurrent unit (GRU) network, and taking an output of the bidirectional GRU network as a context vector of the article;
S 10 : predicting a probability of being a start index and a probability of being an end index of each position in the article separately according to the context vector of the article and the encoded question vector, and taking a result with a maximum probability of being the start index and a maximum probability of being the end index as an answer position in the article to complete the machine reading comprehension; and
S 11 : providing word or words located at the answer position in the article to the user as the answer to the question.

2. The method according to claim 1 , wherein a specific method of step S 1 comprises the following sub-steps:
S 1 - 1 : indicating each word m in the question and the article as a character sequence (c 1 . . . , c |m| ), and indicating each word in a word list as a vector of d c dimension;
S 1 - 2 : applying a convolution kernel with a size of w∈ d c ×w to each word sequence, and adopting the following formula to obtain a feature vector f i :
f i =tan h ( w T c i:i+w−1 +b )

where, tan h(⋅) is a hyperbolic tangent function; c i:i+w−1 is a character sequence segment; b is a deviation parameter; (⋅) T is a transpose of a matrix; and is a number field; and
S 1 - 3 : performing a maximum pooling operation on all feature vectors to obtain the character-level indication vector of the question and the character-level indication vector of the article, respectively.

3. The method according to claim 1 , wherein a specific method of step S 2 comprises the following sub-steps:
S 2 - 1 : obtaining the encoded question vector Q R according to the following formulas:

S

i
:

=

W
s
T

[

P
;
Q
:

P
∘
Q

]

,

Q
′

=

softmax
(

S

i
:

)

·
Q

,

S
=

softmax
(

Q

′
⁢
T

⁢

W
1

⁢

Q
′

)

,

Q
″

=

S
·

Q
′

,

Q
agg

=

tanh
⁡
(

W
f

[

Q
;

Q
″

;

Q
″

-

Q
;

Q
″

∘
Q

]

+

b
f

)

,

b
j

=

exp
⁡
(

w
·

Q
agg

)

Assignees

• Univ Electronic Sci & Tech China

Inventors

• Li Jianping
• Gu Xiaofeng
• Hu Jian
• Sun Ruinan
• Feng Wenting
• Li Shunli
• Jiang Sheng

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F40/216

  using statistical methods · CPC title

• G06F40/20 Primary

  Natural language analysis (semantic analysis of natural language G06F40/30) · CPC title