Method and apparatus for detecting text

Patent metadata

Field	Value
Publication number	US-10762376-B2
Application number	US-201816175590-A
Country	US
Kind code	B2
Filing date	Oct 30, 2018
Priority date	Jan 30, 2018
Publication date	Sep 1, 2020
Grant date	Sep 1, 2020

Abstract

A method and apparatus for detecting text are provided. The method includes: extracting features of a to-be-detected image; predicting using a character detection network a probability of each pixel point in the to-be-detected image being a character pixel point, and position information of each pixel point relative to a bounding box of a character including the pixel point when the pixel point is the character pixel point; determining position information of bounding boxes of candidate characters based on the prediction result of the character detection network; inputting the extracted features into a character map network, converting a feature map outputted by the character map network, and generating character vectors; determining a neighboring candidate character of each candidate character, and connecting each candidate character with an associated neighboring candidate character to form a character set; and determining a character area of the to-be-detected image.

First claim

What is claimed is:

1. A method for detecting text, the method comprising:
extracting features of a to-be-detected image in a plurality of abstract levels using a feature extraction network of a text detection model;
predicting, using a character detection network of the text detection mode based on the extracted features of the to-be-detected image, a probability of each pixel point in the to-be-detected image being a character pixel point, and position information of the each pixel point relative to a bounding box of a character including the each pixel point when the each pixel point is the character pixel point;
determining position information of bounding boxes of candidate characters, based on the probability of the each pixel point being the character pixel point and the position information of the each pixel point relative to the bounding box of the character including the each pixel point when the each pixel point is the character pixel point;
inputting the extracted features of the to-be-detected image into a character mapping network of the text detection model, converting a feature map outputted by the character mapping network based on the position information of the bounding boxes of the candidate characters, and generating character vectors characterizing features of the candidate characters;
determining a neighboring candidate character of each candidate character in the to-be-detected image based on the position information of the bounding boxes of the candidate characters, and connecting the each candidate character with an associated neighboring candidate character to form a character set based on a degree of difference between the each candidate character and the neighboring candidate character, the degree of difference being calculated using the character vectors; and
determining a character area of the to-be-detected image based on the position information of the bounding boxes of the candidate characters in the character set.

2. The method according to claim 1 , wherein the extracting features of a to-be-detected image in a plurality of abstract levels using a feature extraction network of a text detection model comprises:
inputting the to-be-detected image into the feature extraction network to extract outputs from a plurality of different convolutional layers of the feature extraction network, and using the outputs as features in the plurality of abstract levels; and
splicing the features in the plurality of abstract levels, or processing the features in the plurality of abstract levels using a feature pyramid network, to generate the features of the to-be-detected image.

3. The method according to claim 1 , wherein the position information of the each pixel point relative to the bounding box of the character including the each pixel point when the each pixel point is the character pixel point comprises: offsets of coordinates of the each pixel point relative to coordinates of two vertexes on a diagonal line of a rectangular bounding box of the character including the each pixel point when the each pixel point is the character pixel point.

4. The method according to claim 3 , wherein the determining position information of bounding boxes of candidate characters based on the probability of the each pixel point being the character pixel point, and the position information of the each pixel point relative to the bounding box of the character including the each pixel point when the each pixel point is the character pixel point comprises:
determining pixel points having the probability greater than a preset probability threshold as character pixel points;
determining, based on the offsets of the coordinates of each of the determined character pixel points relative to the coordinates of the two vertexes on the diagonal line of the rectangular bounding box of the character including the each character pixel point, coordinates of bounding boxes of characters positioned by the character pixel points; and
filtering out coordinates of a bounding box of a repeatedly positioned character from the coordinates of the bounding boxes of the characters positioned by the character pixel points using a non-maximal value suppression method, to obtain coordinates of the bounding boxes of the candidate characters.

5. The method according to claim 1 , wherein the determining a neighboring candidate character of each candidate character in the to-be-detected image based on the position information of the bounding box of the each candidate character comprises:
classifying the candidate characters using a k-nearest neighbors algorithm based on the position information of the bounding boxes of the candidate characters, and determining the neighboring candidate character of the each candidate character based on a classification result; and
the connecting the each candidate character with the associated neighboring candidate character to form a character set based on a degree of difference between the each candidate character and the neighboring candidate character calculated using the character vector comprises:
calculating a Euclidean distance between a character vector of the each candidate character and a character vector of each of the neighboring candidate character, and using the Euclidean distance as the degree of difference between the each candidate character and the each neighboring candidate character; and
using a neighboring candidate character having the degree of the difference smaller than a preset difference degree threshold as a neighboring candidate character associated with the each candidate character, and connecting the each candidate character with the associated neighboring candidate character to form the character set.

6. The method according to claim 1 , wherein the determining a character area of the to-be-detected image based on the position information of the bounding box of the each candidate character in the character set comprises:
drawing a surrounding line surrounding all characters in the character set to form the character area of the to-be-detected image based on the position information of the bounding boxes of the candidate characters in the character set.

7. The method according to claim 1 , further comprising:
training the text detection model using a machine learning method based on a sample image.

8. The method according to claim 7 , wherein the training the text detection model using a machine learning method based on a sample image comprises:
acquiring the sample image including characters marked with bounding boxes;
inputting the sample image into the text detection model to predict a character area in the sample image, and to obtain a prediction result on whether a pixel point in the sample image is a character pixel point, a prediction result on position information of a bounding box of a character including the character pixel point in the sample image, and a prediction result on a character set in the sample image; and
calculating a value of a preset loss function, calculating a gradient of each parameter in the text detection model relative to the preset loss function, and updating the each parameter of the model using a back propagation algorithm, until the value of the preset loss function meets a preset convergence condition,
wherein the preset loss function comprises a classification loss function, a bounding box regression loss function, and a character connecting loss function;
wherein the value of the classification loss function is used for characterizing a difference between a prediction result of the character detection network on whether the pixel point in the sample image is the character pixel point and a marked result on whether the pixel point in the sample image is the character pi

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Liu Jiaming
• Zhang Chengquan
• Han Junyu
• Ding Errui

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06V30/15

  Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques · CPC title

• G06V30/153

  using recognition of characters or words · CPC title

• G06F18/24147

  Distances to closest patterns, e.g. nearest neighbour classification · CPC title

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title