Method and apparatus for sequence labeling on entity text, and non-transitory computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-11907661-B2
Application number	US-202117455967-A
Country	US
Kind code	B2
Filing date	Nov 22, 2021
Priority date	Nov 25, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

A method and an apparatus for sequence labeling on an entity text, and a non-transitory computer-readable recording medium are provided. In the method, a start position of an entity text within a target text is determined. Then, a first matrix is generated based on the start position of the entity text. Elements in the first matrix indicates focusable weights of each word with respect to other words in the target text. Then, a named entity recognition model is generated using the first matrix. The named entity recognition model is obtained by training using first training data, the first training data includes word embeddings corresponding to respective texts in a training text set, and the texts are texts whose entity label has been labeled. Then, the target text is input to the named entity recognition model, and probability distribution of the entity label is output.

First claim

What is claimed is:

1. A method for sequence labeling on an entity text, the method comprising:
determining a start position of an entity text within a target text, the target text being a text having an entity label to be recognized;
generating a first matrix based on the start position of the entity text within the target text, the number of rows and the number of columns of the first matrix being equal to a sequence length of the target text, elements in the first matrix indicating focusable weights of each word with respect to other words in the target text, the focusable weight of a word in the target text that is within the entity text with respect to a first word being greater than the focusable weight of the word with respect to a second word, the first word including the word, and one or more words between the start position of the entity text and the word, and the second word being a word other than the first word in the target text;
generating a named entity recognition model using the first matrix, the named entity recognition model being obtained by training using first training data, the first training data including word embeddings corresponding to one or more respective texts in a training text set, and the texts in the training text set being texts whose entity label has been labeled; and
inputting the target text to the named entity recognition model, and outputting probability distribution of the entity label corresponding to the target text,
wherein determining the start position of the entity text within the target text includes
performing at least one of data format conversion and data noise removal on the target text to obtain a first text;
segmenting words of the first text to obtain a first word sequence;
segmenting word segments of the first word sequence to divide the first word sequence into a first word segment sequence; and
inputting the first word segment sequence to a second training model to obtain the start position of the entity text.

2. The method for sequence labeling on an entity text as claimed in claim 1 ,
wherein segmenting the word segments of the first word sequence includes
constructing a first initial model, and training the first initial model using second training data to obtain a first segmentation model, the second training data including a plurality of sets of data, and each set of the data including a word sequence and a word segment sequence obtained by segmenting the word sequence; and
segmenting the word segments of the first word sequence using the first segmentation model.

3. The method for sequence labeling on an entity text as claimed in claim 1 , the method further comprising:
obtaining the second training model,
wherein obtaining the second training model includes
constructing a second initial model, and training the second initial model using third training data to obtain the second training model, the third training data including a plurality of word segment sequences whose start position of entity text has been labeled.

4. The method for sequence labeling on an entity text as claimed in claim 1 ,
wherein the element in the i-th row and the j-th column in the first matrix indicates the focusable weight of the i-th word with respect to the j-th word in the target text, and
wherein the elements in the a-th row in the first matrix corresponds to the a-th word in the target text, and if the a-th word and the k words on the left of the a-th word are within the entity text, values of the a-k-th element to the a-th element in the a-th row are set to a non-zero value, and values of other elements in the a-th row are set to zero, where a, i, j and k are positive integers.

5. The method for sequence labeling on an entity text as claimed in claim 1 ,
wherein the named entity recognition model is a self-attention model, and the named entity recognition model includes an attention layer, a feed forward neural network layer, and a span focus layer, a linear layer, and a softmax layer,
wherein input of the attention layer is word embeddings corresponding to the target text, and output of the attention layer is input of the feed forward neural network layer,
wherein output of the feed forward neural network layer is input of the span focus layer, and output of the span focus layer is

Attention
⁢

⁢

(

Q
,
K
,
V

)

=

softmax
⁢

⁢

(

Q
⁢

K
T

d

k
⁢
1

)

*
Mas
⁢

k
span

*
V

where Q=W Q *H1, K=W K *H1, V=W V *H1, H1 is the output of the feed forward neural network layer, W Q , W K and W V are trainable parameter matrices, Mask span is the first matrix, and d ki is a dimension of an H1 matrix,
wherein the linear layer converts a dimension of the output of the span focus layer, and
wherein the softmax layer normalizes an output result.

6. An apparatus for sequence labeling on an entity text, the apparatus comprising:
a memory storing computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions such that the one or more processors are configured to
determine a start position of an entity text within a target text, the target text being a text having an entity label to be recognized;
generate a first matrix based on the start position of the entity text within the target text, the number of rows and the number of columns of the first matrix being equal to a sequence length of the target text, elements in the first matrix indicating focusable weights of each word with respect to other words in the target text, the focusable weight of a word in the target text that is within the entity text with respect to a first word being greater than the focusable weight of the word with respect to a second word, the first word including the word, and one or more words between the start position of the entity text and the word, and the second word being a word other than the first word in the target text;
generate a named entity recognition model using the first matrix, the named entity recognition model being obtained by training using first training data, the first training data including word embeddings corresponding to one or more respective texts in a training text set, and the texts in the training text set being texts whose entity label has been labeled; and
input the target text to the named entity recognition model, and output probability distribution of the entity label corresponding to the target text,
wherein determining the start position of the entity

Assignees

• Tong Yixuan
• Zhang Yongwei
• Dong Bin
• Jiang Shanshan
• Zhang Jiashi
• Ricoh Co Ltd

Inventors

• Tong Yixuan
• Zhang Yongwei
• Dong Bin
• Jiang Shanshan
• Zhang Jiashi

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F40/295 Primary

  Named entity recognition · CPC title

• G06F40/216

  using statistical methods · CPC title

• G06F16/3344

  using natural language analysis · CPC title