Method, apparatus and device for recognizing text type

Patent metadata

Field	Value
Publication number	US-11281860-B2
Application number	US-201816160950-A
Country	US
Kind code	B2
Filing date	Oct 15, 2018
Priority date	Aug 31, 2016
Publication date	Mar 22, 2022
Grant date	Mar 22, 2022

Abstract

The present disclosure discloses a method and apparatus for recognizing a text type, and a device. A specific embodiment of the method comprises: preprocessing a pre-acquired text to obtain a keyword set of the text; calculating an occurrence probability value of each keyword in the keyword set in the text; importing, for the each keyword in the keyword set, the keyword and the occurrence probability value corresponding to the keyword into a pre-established document topic generation model, and determining an occurrence probability value of each topic in the text preset in the document topic generation model; and recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text. The embodiment improves an accuracy of recognizing the type of the text.

First claim

What is claimed is:

1. A method for recognizing a text type, comprising:
preprocessing a pre-acquired text to obtain a keyword set of the text;
calculating an occurrence probability value of each keyword in the keyword set in the text, the occurrence probability value of the keyword being a ratio of an occurrence frequency of the keyword in the text to a total number of words in the text, or a ratio of the occurrence frequency of the keyword in the text to a number of keywords in the keyword set of the text;
importing, for the each keyword in the keyword set, the keyword and the occurrence probability value corresponding to the keyword into a pre-established document topic generation model, and determining an occurrence probability value of each topic in the text preset in the document topic generation model, the document topic generation model being used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and both the occurrence probability value of the keyword in the text and a pre-acquired occurrence probability value of the keyword in the each topic, wherein the occurrence probability value of the keyword in the text equals to a sum of each product of the occurrence probability value of the each topic in the text and the occurrence probability value of the keyword in the each topic; and
recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text, comprising: defining a type of the topic having a largest occurrence probability value as the type to which the text belongs,
wherein the type includes a positive class and a negative class; and
the recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text comprises:
importing the occurrence probability value of the each topic in the text into a pre-established first logistic regression model, to obtain a first probability value of the text belonging to the positive class, wherein the first logistic regression model is used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and the first probability value of the text belonging to the positive class; and
recognizing whether the text belongs to the positive class based on the first probability value.

2. The method according to claim 1 , wherein the recognizing whether the text belongs to the positive class based on the first probability value comprises:
importing, in response to the first probability value being less than a preset first threshold, the occurrence probability value of the each topic in the text into a pre-established second logistic regression model, to obtain a second probability value of the text belonging to the positive class, wherein the second logistic regression model is used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and the second probability value of the text belonging to the positive class, a regression parameter of the second logistic regression model is different from a regression parameter of the first logistic regression model, and the regression parameter is used to represent a probability of the each topic belonging to the positive class; and
determining, in response to the second probability value being greater than a preset second threshold, the text belonging to the positive class.

3. The method according to claim 2 , wherein the recognizing whether the text belongs to the positive class based on the first probability value further comprises:
determining, in response to the second probability value being less than the preset second threshold, the text belonging to the negative class.

4. The method according to claim 1 , wherein the recognizing whether the text belongs to the positive class based on the first probability value further comprises:
determining, in response to the first probability value being greater than the preset first threshold, the text belonging to the positive class.

5. The method according to claim 1 , wherein the preprocessing a pre-acquired text to obtain a keyword set of the text comprises:
removing a special symbol in the text;
performing word segmentation on the text after the special symbol is removed, to obtain a word set; and
removing a stop word in the word set to obtain the keyword set.

6. An apparatus for recognizing a text type, comprising:
at least one processor; and
a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
preprocessing a pre-acquired text to obtain a keyword set of the text;
calculating an occurrence probability value of each keyword in the keyword set in the text, the occurrence probability value of the keyword being a ratio of an occurrence frequency of the keyword in the text to a total number of words in the text, or a ratio of the occurrence frequency of the keyword in the text to a number of keywords in the keyword set of the text;
importing, for the each keyword in the keyword set, the keyword and the occurrence probability value corresponding to the keyword into a pre-established document topic generation model, and determining an occurrence probability value of each topic in the text preset in the document topic generation model, the document topic generation model being used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and both the occurrence probability value of the keyword in the text and a pre-acquired occurrence probability value of the keyword in the each topic, wherein the occurrence probability value of the keyword in the text equals to a sum of each product of the occurrence probability value of the each topic in the text and the occurrence probability value of the keyword in the each topic; and
recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text, comprising: defining a type of the topic having a largest occurrence probability value as the type to which the text belongs,
wherein the type includes a positive class and a negative class; and
the recognizing a type to which the text belongs based on the occurrence probability value of the each topic in the text comprises:
importing the occurrence probability value of the each topic in the text into a pre-established first logistic regression model, to obtain a first probability value of the text belonging to the positive class, wherein the first logistic regression model is used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and the first probability value of the text belonging to the positive class; and
recognizing whether the text belongs to the positive class based on the first probability value.

7. The apparatus according to claim 6 , wherein the recognizing whether the text belongs to the positive class based on the first probability value comprises:
importing, in response to the first probability value being less than a preset first threshold, the occurrence probability value of the each topic in the text into a pre-established second logistic regression model, to obtain a second probability value of the text belonging to the positive class, wherein the second logistic regression model is used to represent a corresponding relationship between the occurrence probability value of the each topic in the text and the second probability value of the text belonging to the positive class, a regression parameter of the second logistic regression model is different from a regression parameter of the fi

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Yue Aizhen
• Cui Yan
• Zhao Hui
• Gao Xian
• Wang Sijiang
• Tan Jing

Classifications

• G06F40/279 Primary

  Recognition of textual entities · CPC title

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F40/216 Primary

  using statistical methods · CPC title