System and Method for Extracting Table Data from Text Documents Using Machine Learning

Patent metadata

Field	Value
Publication number	US-2016104077-A1
Application number	US-201514879349-A
Country	US
Kind code	A1
Filing date	Oct 9, 2015
Priority date	Oct 10, 2014
Publication date	Apr 14, 2016
Grant date	—

Abstract

Systems and methods for extracting table data from text documents using machine learning are provided. The systems and methods comprise electronically receiving at a computer system a document having one or more tables, each table having one or more whitespace features, processing the document using a first computer model executed by the computer system to classify each row of the one or more tables as a header row or a data row, processing the document using a second computer model executed by the computer system to classify each whitespace feature in each row conditional on classification of each row by the first computer model, the second computer model identifying whether a whitespace feature corresponds to information missing from the one or more tables, and generating an output of the classified whitespace features and storing the output in a digital file.

First claim

What is claimed is:

1 . A method for electronically extracting table data from text documents using machine learning, comprising:
electronically receiving at a computer system a document having one or more tables, each table having one or more whitespace features; processing the document using a first computer model executed by the computer system to classify each row of the one or more tables as a header row or a data row; processing the document using a second computer model executed by the computer system to classify each whitespace feature in each row conditional on classification of each row by the first computer model, the second computer model identifying whether a whitespace feature corresponds to information missing from the one or more tables; and generating an output of the classified whitespace features and storing the output in a digital file.

2 . The method of claim 1 , wherein the first computer model comprises a random fields classifier.

3 . The method of claim 2 , wherein the random fields classifier is trained using a set of training tables.

4 . The method of claim 1 , wherein the second computer model comprises a multinomial logistic classifier.

5 . The method of claim 4 , wherein the multinomial logistic classifier is trained using a set of training tables.

6 . The method of claim 1 , wherein the information missing comprises a missing cell.

7 . A non-transitory computer-readable medium having computer-readable instructions stored thereon which, when executed by a computer system, cause the computer system to perform the steps of:
electronically receiving at a computer system a document having one or more tables, each table having one or more whitespace features; processing the document using a first computer model executed by the computer system to classify each row of the one or more tables as a header row or a data row; processing the document using a second computer model executed by the computer system to classify each whitespace feature in each row conditional on classification of each row by the first computer model, the second computer model identifying whether a whitespace feature corresponds to information missing from the one or more tables; and generating an output of the classified whitespace features and storing the output in a digital file.

8 . The non-transitory computer-readable medium of claim 7 , wherein the first computer model comprises a random fields classifier.

9 . The non-transitory computer-readable medium of claim 8 , wherein the random fields classifier is trained using a set of training tables.

10 . The non-transitory computer-readable medium of claim 7 , wherein the second computer model comprises a multinomial logistic classifier.

11 . The non-transitory computer-readable medium of claim 10 , wherein the multinomial logistic classifier is trained using a set of training tables.

12 . The non-transitory computer-readable medium of claim 7 , wherein the information missing comprises a missing cell.

13 . A system for electronically extracting table data from text documents using machine learning, comprising:
a computer system for electronically receiving a document having one or more tables, each table having one or more whitespace features; an engine executed by the computer system, the engine:
processing the document using a first computer model executed by the computer system to classify each row of the one or more tables as a header row or a data row;
processing the document using a second computer model executed by the computer system to classify each whitespace feature in each row conditional on classification of each row by the first computer model, the second computer model identifying whether a whitespace feature corresponds to information missing from the one or more tables; and
generating an output of the classified whitespace features and storing the output in a digital file.

14 . The system of claim 13 , wherein the first computer model comprises a random fields classifier.

15 . The system of claim 14 , wherein the random fields classifier is trained using a set of training tables.

16 . The system of claim 13 , wherein the second computer model comprises a multinomial logistic classifier.

17 . The system of claim 16 , wherein the multinomial logistic classifier is trained using a set of training tables.

18 . The system of claim 13 , wherein the information missing comprises a missing cell.

Assignees

• Univ Columbia

Inventors

• Jackson Jr Robert J
• Mitts Joshua R
• Zhang Jing

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F40/177

  of tables; using ruled lines · CPC title

• G06F40/163

  Handling of whitespace · CPC title

• G06N99/005 Primary

  Physics · mapped topic

• G06F17/30011

  Physics · mapped topic