Method and apparatus for processing information

Patent metadata

Field	Value
Publication number	US-11775776-B2
Application number	US-202117147881-A
Country	US
Kind code	B2
Filing date	Jan 13, 2021
Priority date	Jan 14, 2020
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

A method and an apparatus for processing information are provided. The method can include: acquiring a word sequence obtained by performing word segmentation on two paragraphs in a text; inputting the word sequence into a to-be-trained natural language processing model to generate a word vector corresponding to a word in the word sequence; inputting the word vector into a preset processing layer of the to-be-trained natural language processing model; predicting whether the two paragraphs are adjacent, and a replaced word in the two paragraphs; and acquiring reference information of the two paragraphs, and training the to-be-trained natural language processing model to obtain a trained natural language processing model, based on the prediction result and the reference information.

First claim

What is claimed is:

1. A method for processing information, the method comprising:
acquiring a word sequence obtained by performing word segmentation on two paragraphs in a text, wherein the word sequence comprises at least one specified identifier for replacing a first word;
inputting the word sequence into a to-be-trained natural language processing model to generate a word vector corresponding to a second word in the word sequence, wherein the word vector is used to represent the second word in the word sequence and a position of the second word;
inputting the word vector into a preset processing layer of the to-be-trained natural language processing model, wherein the preset processing layer comprises an encoder and a decoder;
predicting whether the two paragraphs are adjacent, and the replaced first word in the two paragraphs, to obtain a prediction result, based on a processing result output by the preset processing layer;
acquiring reference information of the two paragraphs, and training the to-be-trained natural language processing model to obtain a trained natural language processing model, based on the prediction result and the reference information, wherein the reference information comprises adjacent information indicating whether the two paragraphs are adjacent, and the replaced first word;
acquiring first sample information, wherein the first sample information comprises a first paragraph word sequence obtained by performing word segmentation on a first target paragraph, and a first specified attribute;
inputting the first sample information comprising the first paragraph word sequence and the first specified attribute into the trained natural language processing model to predict correlation information, wherein the correlation information is used to indicate a correlation value between the first paragraph word sequence and the first specified attribute, and the correlation value is determined by determining whether the first specified attribute or a meaning of the first specified attribute is included in the first paragraph word sequence; and
training the trained natural language processing model to obtain a first model, based on predicted correlation information and correlation information for labeling the first sample information, wherein a loss value is calculated based on the correlation information for labeling the first sample information and the predicted correlation information to implement further training of the trained natural language processing model.

2. The method according to claim 1 , wherein the method further comprises:
acquiring second sample information, wherein the second sample information comprises a second paragraph word sequence obtained by performing word segmentation on a second target paragraph, and a second specified attribute, wherein an attribute matching the second specified attribute is comprised in the second paragraph word sequence, and the attribute matching the second specified attribute completely matches or partially matches the second specified attribute;
inputting the second sample information into the trained natural language processing model, and predicting an attribute value of the second specified attribute in the second paragraph word sequence; and
training the trained natural language processing model to obtain a second model, based on the predicted attribute value and an attribute value for labeling the attribute matching the second specified attribute.

3. The method according to claim 2 , wherein the predicting an attribute value of the second specified attribute in the second paragraph word sequence, comprises:
predicting position information of the attribute value of the second specified attribute in the second paragraph word sequence, wherein the position information comprises start position information and end position information.

4. The method according to claim 2 , wherein the method further comprises:
acquiring a text word sequence obtained by performing word segmentation on a target text, and dividing the text word sequence into a plurality of paragraph word sequences;
determining paragraph word sequences related to a target attribute from the plurality of paragraph word sequences;
inputting the target attribute and the paragraph word sequences related to the target attribute into the first model, and predicting correlation information between the target attribute and each of the paragraph word sequences related to the target attribute, wherein the correlation information comprises a correlation value;
selecting a preset number of paragraph word sequence from the plurality of paragraph word sequences related to the target attribute in a descending order of the correlation values;
inputting the target attribute and the preset number of paragraph word sequence into the second model, and predicting an attribute value of the target attribute and a confidence level of the attribute value of the target attribute; and
determining an attribute value of the target attribute from the predicted attribute value of the target attribute, based on the correlation value and the confidence level.

5. The method according to claim 4 , wherein the determining an attribute value of the target attribute from the predicted attribute value of the target attribute, based on the correlation value and the confidence level, comprises:
determining, for each of the predicted attribute value of the target attribute, a product of a correlation value between a paragraph word sequence where the each attribute value is located and the target attribute, and a confidence level of the each attribute value; and
determining an attribute value corresponding to a maximum product from the predicted attribute value of the target attribute as the attribute value of the target attribute.

6. The method according to claim 4 , wherein the determining paragraph word sequences related to a target attribute from the plurality of paragraph word sequences, comprises:
determining, for each paragraph word sequence of the plurality of paragraph word sequences, whether a third word matching the target attribute is comprised in the each paragraph word sequence; and
determining the each paragraph word sequence as the paragraph word sequence related to the target attribute, if the third word matching the target attribute is comprised in the each paragraph word sequence.

7. The method according to claim 1 , wherein the inputting the word sequence into a to-be-trained natural language processing model to generate a word vector corresponding to a second word in the word sequence, comprises:
inputting the word sequence into an embedding layer of the to-be-trained natural language processing model;
converting, for the second word in the word sequence, the second word into an identifier of the second word through the embedding layer, and converting the identifier of the second word into a first vector;
converting position information of the second word in the word sequence into a second vector through the embedding layer;
determining paragraph position information indicating a paragraph in which the second word is located in the two paragraphs through the embedding layer, and converting the paragraph position information into a third vector; and
splicing the first vector, the second vector, and the third vector to obtain a word vector corresponding to the second word.

8. The method according to claim 1 , wherein the preset processing layer comprises a plurality of cascaded preset processing layers; and the inputting the word vector into a preset processing layer of the to-be-trained natural language processing model, comprises:
inputting the word vector into a first preset processing layer of the plurality of cascaded pres

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Li Shuangjie
• Yu Miao
• Shi Yabing
• Hao Xuefeng
• Song Xunchao
• Jiang Ye
• Zhang Yang
• Zhu Yong

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F40/40 Primary

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/289

  Phrasal analysis, e.g. finite state techniques or chunking · CPC title