Method and apparatus for detecting table, device and storage medium

Patent metadata

Field	Value
Publication number	US-12154359-B2
Application number	US-202217743687-A
Country	US
Kind code	B2
Filing date	May 13, 2022
Priority date	Aug 12, 2021
Publication date	Nov 26, 2024
Grant date	Nov 26, 2024

Abstract

A method and apparatus for detecting a table. The method includes: acquiring a to-be-processed image; inputting the to-be-processed image into a pre-trained deep learning model, and outputting a full table detection branch result, a column detection branch result and a header detection branch result through the deep learning model; where the full table detection branch result represents a detection result for a full table in the to-be-processed image, the column detection branch result represents a detection result for a column in the table in the to-be-processed image, and the header detection branch result represents a detection result for a header in the to-be-processed image; and obtaining a detection result of the table in the to-be-processed image, based on the full table detection branch result, the column detection branch result and the header detection branch result.

First claim

What is claimed is:

1. A method for detecting a table, the method comprising:
acquiring a to-be-processed image;
inputting the to-be-processed image into a pre-trained deep learning model, and outputting a full table detection branch result, a column detection branch result and a header detection branch result through the deep learning model; wherein the full table detection branch result represents a detection result for a full table in the to-be-processed image, the column detection branch result represents a detection result for a column in the table in the to-be-processed image, and the header detection branch result represents a detection result for a header in the to-be-processed image; and
obtaining a detection result of the table in the to-be-processed image, based on the full table detection branch result, the column detection branch result and the header detection branch result by: superimposing the full table detection branch result, the column detection branch result and the header detection branch result; and fitting a boundary range to a result obtained by the superposition, to obtain a detection frame for the table in the to-be-processed image.

2. The method according to claim 1 , wherein the full table detection branch result comprises a full table detection branch binary map, and the full table detection branch binary map indicates whether each pixel in the to-be-processed image belongs to the full table; the column detection branch result comprises a column detection branch binary map, the column detection branch binary map indicates whether each pixel in the to-be-processed image belongs to the column in the table; and the header detection branch result comprises a header detection branch binary map, and the header detection branch binary map indicates whether each pixel in the to-be-processed image belongs to the header;
wherein superimposing the full table detection branch result, the column detection branch result and the header detection branch result, comprises:
determining a union of the full table detection branch binary map, the column detection branch binary map, and the header detection branch binary map to obtain a union binary map; and
wherein fitting the boundary range to the result obtained by the superposition, to obtain the detection frame for the table in the to-be-processed image, comprises:
determining a connected region in the union binary map; and
determining an enclosing rectangle of the connected region, wherein the enclosing rectangle is the detection frame for the table in the to-be-processed image.

3. The method according to claim 1 , wherein the method further comprises:
acquiring a plurality of sample images and a marked full table, a marked column, and a marked header of a table in each sample image;
inputting, for each sample image, the sample image into an initial model;
performing feature extraction on the sample image by using the initial model;
inputting a result of the feature extraction into a full table detection branch, a column detection branch and a header detection branch respectively, to obtain a sample full table detection branch result, a sample column detection branch result and a sample header detection branch result; and
training the initial model using the sample full table detection branch result, the sample column detection branch result and the sample header detection branch result, as well as the marked full table, the marked column, and the marked header of the table, to obtain the deep learning model.

4. The method according to claim 3 , wherein training the initial model using the sample full table detection branch result, the sample column detection branch result and the sample header detection branch result, as well as the marked full table, the marked column, and the marked header of the table, to obtain the deep learning model, comprises:
calculating a first loss function value using the sample full table detection branch result and the marked full table of the table, using a first loss function;
calculating a second loss function value using the sample column detection branch result and the marked column, using a second loss function;
calculating a third loss function value using the sample header detection branch result and the marked header, using a third loss function;
summing the first loss function value, the second loss function value and the third loss function value to obtain a total function value; and
training the initial model based on the total function value to obtain the deep learning model.

5. The method according to claim 1 , wherein the method further comprises:
acquiring a plurality of sample images and a marked full table, a marked column, and a marked header of a table in each sample image;
inputting, for each sample image, the sample image into an initial model;
performing feature extraction on the sample image by using the initial model;
inputting a result of the feature extraction into a full table detection branch, a column detection branch and a header detection branch respectively, to obtain a sample full table detection branch result, a sample column detection branch result and a sample header detection branch result; and
training the initial model using the sample full table detection branch result, the sample column detection branch result and the sample header detection branch result, as well as the marked full table, the marked column, and the marked header of the table, to obtain the deep learning model.

6. The method according to claim 2 , wherein the method further comprises:
acquiring a plurality of sample images and a marked full table, a marked column, and a marked header of a table in each sample image;
inputting, for each sample image, the sample image into an initial model;
performing feature extraction on the sample image by using the initial model;
inputting a result of the feature extraction into a full table detection branch, a column detection branch and a header detection branch respectively, to obtain a sample full table detection branch result, a sample column detection branch result and a sample header detection branch result; and
training the initial model using the sample full table detection branch result, the sample column detection branch result and the sample header detection branch result, as well as the marked full table, the marked column, and the marked header of the table, to obtain the deep learning model.

7. An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring a to-be-processed image;
inputting the to-be-processed image into a pre-trained deep learning model, and outputting a full table detection branch result, a column detection branch result and a header detection branch result through the deep learning model; wherein the full table detection branch result represents a detection result for a full table in the to-be-processed image, the column detection branch result represents a detection result for a column in the table in the to-be-processed image, and the header detection branch result represents a detection result for a header in the to-be-processed image; and
obtaining a detection result of the table in the to-be-processed image, based on the full table detection branch result, the column detection branch result and the header detection branch result by: superimposing the full table detection branch result, the column detection branch result and the header detection branch result; and

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Zhou Xia
• Wang Leyi
• Li Qiaoyi
• Qin Duohao
• Liu Minghao

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title