Method, apparatus and device for extracting information

Patent metadata

Field	Value
Publication number	US-11288593-B2
Application number	US-201715829677-A
Country	US
Kind code	B2
Filing date	Dec 1, 2017
Priority date	Oct 23, 2017
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

A method, apparatus and device are for extracting information. The method includes: acquiring an annotated corpus, which includes a plurality of sample sentences and annotated information sets corresponding to the sample sentences, constructing an input sequence and an output sequence based on the sample sentences and the annotated information sets corresponding to the sample sentences, obtaining an information extraction model generating the output sequence from the input sequence by carrying out training with a deep learning method, and inputting a to-be-processed sentence into the information extraction model to extract a knowledge information set included in the to-be-processed sentence. The annotated information set includes information of at least one piece of the following types of knowledge to be extracted from corresponding sample sentences: knowledge based on verbs or prepositions, knowledge based on noun attributes, knowledge of entity description, and knowledge of a relationship between an entity and a concept.

First claim

What is claimed is:

1. A method for extracting information, the method comprising:
acquiring an annotated corpus, the annotated corpus comprising a plurality of sample sentences and annotated information sets corresponding to the sample sentences;
constructing an input sequence and a corresponding output sequence based on the sample sentences and the annotated information sets corresponding to the sample sentences in the annotated corpus respectively;
obtaining an information extraction model generating the output sequence from the input sequence by carrying out training with a deep learning method; and
inputting a to-be-processed sentence into the generated information extraction model to extract a knowledge information set included in the to-be-processed sentence,
wherein the annotated information set comprises information, standardized according to a preset rule, of at least one piece of following knowledge to be extracted from corresponding sample sentences:
knowledge based on verbs or prepositions, knowledge based on noun attributes, knowledge of entity description, and knowledge of a relationship between an entity and a concept, the preset rule being a pre-defined symbol-assisted expression of various types of knowledge in the sample sentences,
wherein obtaining an information extraction model generating the output sequence from the input sequence by carrying out training with a deep learning method comprises:
carrying out vectorization processing on objects in the input sequence and annotated information in the output sequence;
constructing a neural network of the information extraction model, the neural network of the information extraction model comprising an encoder and a decoder;
mapping the input sequence after the vectorization processing into an input hidden state sequence with the encoder, and mapping the output sequence into an output hidden state sequence with the decoder;
decoding the input hidden state sequence with the decoder to obtain a predictive state sequence; and
adjusting parameters of the neural network according to a difference between the predictive state sequence and the output hidden state sequence; and

wherein input hidden state sequence with the decoder to obtain a predictive state sequence comprises:
for a target position in the predictive state sequence to be generated, obtaining a state, after a predictive state of a previous position of the target position is obtained by decoding, of a hidden layer of the decoder as a current hidden state of the decoder;
extracting a semantic dependence relationship tree in the sample sentence using a natural language processing method;
adding a counter edge for each edge in the semantic dependence relationship tree to construct a path finding map;
finding a semantic path between any two objects in the input sequence in the path finding map, and encoding the semantic path into a semantic vector;
transforming the semantic vector to obtain a semantic dependence relationship attention vector;
by carrying out a third nonlinear transformation based on the current hidden state, the input hidden state and the semantic dependence relationship attention vector, computing a matching degree between each input hidden state in the input hidden state sequence and a predictive state of the target position in the predictive state sequence to be generated;
computing an attention weight of the each input hidden state to the predictive state of the target position based on the matching degree;
carrying out weighted summation on the each input hidden state according to the attention weight to obtain a context vector;
computing a distribution of a probability of the predictive state of the target position based on the context vector, the output hidden state of the previous position of the target position in the output hidden state sequence, and a state of the hidden layer of the decoder when decoding the predictive state of the target position; and
determining the predictive state of the target position according to the probability distribution.

2. The method according to claim 1 , wherein the computing a matching degree between each input hidden state in the input hidden state sequence and a predictive state of the target position in the predictive state sequence to be generated based on the current hidden state comprises:
carrying out a first nonlinear transformation on the current hidden state and the input hidden state to obtain the matching degree between the input hidden state and the predictive state of the target position.

3. The method according to claim 1 , wherein the computing a matching degree between each input hidden state in the input hidden state sequence and a predictive state of the target position in the predictive state sequence to be generated based on the current hidden state comprises:
computing a coverage vector corresponding to the target position based on the current hidden state, wherein the coverage vector is used for recording extracted information in the sample sentence; and
carrying out a second nonlinear transformation on the current hidden state, the input hidden state and the coverage vector corresponding to the previous position of the target position to obtain the matching degree between the input hidden state and the predictive state of the target position.

4. The method according to claim 1 , wherein the probability of the predictive state of the target position is a sum of a probability of copying a word from a corresponding sample sentence as an object in the output sequence and a probability of selecting a target symbol from a preset symbol set and using an object characterized by the target symbol as an object in the output sequence; and
symbols in the preset symbol set are used for combining with the words in the sample sentence, so as to completely characterize one of the following knowledge in the sample sentence: knowledge based on verbs or prepositions, knowledge based on noun attributes, knowledge of entity description, and knowledge of a relationship between an entity and a concept.

5. The method according to claim 4 , wherein the symbols in the preset symbol set comprise:
a symbol for characterizing a separated predicate in a sentence;
a symbol for characterizing reduced information in knowledge expressed by a reduced expression in the sentence;
a symbol for characterizing implicit knowledge in the sentence; and
a symbol for characterizing lost information in the sentence.

6. An apparatus for extracting information, the apparatus comprising:
at least one processor; and
a memory storing instructions, which when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring an annotated corpus, the annotated corpus comprising a plurality of sample sentences and annotated information sets corresponding to the sample sentences;
constructing an input sequence and a corresponding output sequence based on the sample sentences and the annotated information sets corresponding to the sample sentences in the annotated corpus respectively;
generating an information extraction model of the output sequence from the input sequence by carrying out training with a deep learning method; and
inputting a to-be-processed sentence into the generated information extraction model to extract a knowledge information set included in the to-be-processed sentence;
wherein the annotated information set comprises information, standardized according to a preset rule, of at least one piece of the following knowledge to be extracted from corresponding sample sentences:
knowledge based on verbs or prepositions, knowledge based on noun attributes, knowledge of entity description, and knowledge of a relationship betwe

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Sun Mingming
• Li Ping

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title