Character detection method and apparatus

Patent metadata

Field	Value
Publication number	US-10769484-B2
Application number	US-201715419958-A
Country	US
Kind code	B2
Filing date	Jan 30, 2017
Priority date	Dec 30, 2016
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

Disclosed embodiments relate to a character detection method and apparatus. In some embodiments, the method includes: using an image including an annotated word as an input to a machine learning model; selecting, based on a predicted result of characters inside an annotation region of the annotated word predicted and annotation information of the annotated word, characters for training the machine learning model from the characters inside the annotation region of the annotated word predicted; and training the machine learning model based on features of the selected characters. This implementation manner implements the full training of a machine learning model by using existing word level annotated images, to obtain a machine learning model capable of detecting characters in images, thereby reducing the costs for the training of a machine learning model capable of detecting characters in images.

First claim

What is claimed is:

1. A character detection method, comprising:
using an image including an annotated word as an input to a machine learning model, comprising: using a word level annotated image in a word level annotated dataset as the image including the annotated word, the word level annotated image comprising an annotation box surrounding the word for annotating the position of the word;
selecting, based on a predicted result for characters being inside an annotation region of the annotated word and predicted by the machine learning model and annotation information of the annotated word, characters for training the machine learning model from the characters being inside the annotation region of the annotated word and predicted by the machine learning model, wherein the predicted result comprises: bounding boxes corresponding to the characters being inside the annotation region of the annotated word and confidence levels corresponding to the characters being inside the annotation region of the annotated word, and the annotation information comprises a bounding box corresponding to the annotated word;
training the machine learning model based on features of the selected characters; and
detecting characters in an image by using the trained machine learning model,
wherein the selecting characters for training the machine learning model from the characters being inside the annotation region of the annotated word and predicted by the machine learning model comprises:
calculating k neighbors for the bounding boxes corresponding to the characters being inside the annotation region of the annotated word and predicted by the machine learning model, to obtain a connection relationship between the characters, wherein each of the characters is connected to k other characters;
calculating a weight w ij between two connected characters by using the following formula:

w
ij

=

exp
⁡

(

-

d
⁡

(

i
,
j

)

d
_

)

·

(

t
i

+

t
j

)

wherein the two connected characters constitute a character connection pair, d(i, j) represents a distance between the two connected characters, d represents an average distance between characters in all character connection pairs, and t i and t j represent respective confidence levels corresponding to the two connected characters;
finding a maximum spanning tree, wherein the maximum spanning tree comprises sequentially connected characters predicted by the machine learning model, and the sum of the weights between the characters is the greatest;
executing the following selection operation:
pruning each character connection pair in a current tree to obtain multiple subtrees, wherein when the selection operation is executed for the first time, the current tree is the maximum spanning tree;
calculating a score s of a subtree or the current tree by using the following formula:

s
=

w
·
s

⁢

⁢
1

+

(

1
-
w

)

·
s

⁢

⁢
2

s
⁢

⁢
1

=

area
⁡

(

B
chars

)

area
⁡

(

B
anno

)

s
⁢

⁢
2

=

1
-

λ
2

λ
1

wherein B chars represents a bounding box corresponding to a character in the subtree or the current tree, B anno represents the bounding box corresponding to the annotated word, area(B chars ) represents the area of the bounding boxes corresponding to the characters in the subtree or the current tree, area(B anno ) represents the area of the bounding box corresponding to the annotated word, λ 1 and λ 2 respectively represent the greatest feature value and the second greatest feature value of a covariance matrix of the center coordinates of B chars , w is a preset weight when the selection operation is executed for the first time, and w is the weight between the two characters in the character connection pair corresponding to the subtree when the selection operation is not executed for the first time;
determining whether the greatest of the scores of the subtrees is greater than the score of the current tree; and
if yes, using the subtree with the greatest score as the current tree, and executing the selection operation again; or
if not, using the characters in the current tree as the characters for training the machine learning model.

2. The method according to claim 1 , wherein the machine learning model comprises a convolutional neural network.

3. The method according to claim 1 , wherein after the using the image including the annotated word as input to

Assignees

• Baidu online network technology beijing co ltd

Inventors

• Zhang Chengquan
• Hu Han
• Luo Yuxuan
• Han Junyu
• Ding Errui

Classifications

• G06V30/1916

  Validation; Performance evaluation · CPC title

• G06V30/18057

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V30/18181

  Graphical representation, e.g. directed attributed graph · CPC title

• G06V30/15

  Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques · CPC title

• G06V30/19173

  Classification techniques · CPC title