Identification of key segments in document images

Patent metadata

Field	Value
Publication number	US-10699112-B1
Application number	US-201816146562-A
Country	US
Kind code	B1
Filing date	Sep 28, 2018
Priority date	Sep 28, 2018
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

A system and method of automatically learning new keywords in a document image based on context such as when a never before seen keyword exists surrounded by other key-value pairs. A machine learning based approach leverages subword embeddings and two-dimensional geometric contexts in a gradient boosted trees classifier. Keys may be composed of multi-word strings or single-word strings.

First claim

What is claimed is:

1. A computerized method for identifying keywords in a document image, comprising:
(i) retrieving a document image from a set of document images where each document in the set of document images contains information organized in a two-dimensional structure and contains keywords, where each keyword of a set of the keywords has a value associated therewith;
(ii) processing the document image to identify text segments contained within the document image;
(iii) processing the text segments to identify subword embeddings associated with each of the text segments, wherein each of the subword embeddings associated with a text segment represents a character group in the document image,
(iv) generating an n-dimensional vector for each text segment from its subword embeddings;
(v) for each identified text segment, mapping one or more of the n-dimensional vectors to each of the identified text segments to generate for each identified text segment, a feature vector which describes a local context of the identified text segment;
(vi) retrieving an annotated version of the document image containing a visual indication annotation associated with each visual indication of a keyword in the document;
(vii) associating with each visual indication of a keyword in the annotated version of the document image a corresponding feature vector to generate a training document; and
(viii) repeating steps (i) through (vii) for each document from the set of document images to generate a set of training documents.

2. The computerized method of claim 1 wherein each of the subword embeddings utilize a vector representation of one or more n-character groupings of a word, where n is a preselected integer and where a word is represented by a sum of the vector representations.

3. The computerized method of claim 1 wherein step (v) comprises:
selecting for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment.

4. The computerized method of claim 1 wherein step (v) comprises:
selecting for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment, and which overlap the identified text segment by greater than a preselected overlap amount.

5. The computerized method of claim 1 wherein step (v) comprises:
selecting for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment, and wherein the feature vector comprises a concatenation of vectors corresponding to the identified text segment and of a vector corresponding to each of the text segments that are positioned above, below, to the left and to the right of the identified text segment.

6. The computerized method of claim 1 further comprising providing the set of training documents to a supervised learning engine to generate a trained model.

7. A document processing system comprising:
data storage for storing a set of document images where each document in the set of document images contains information organized in a two-dimensional structure and contains keywords, where each keyword of a set of the keywords has a value associated therewith; and
a processor operatively coupled to the data storage and configured to execute instructions that when executed cause the processor to generate a set of training documents from at least a portion of the documents in the set of document images by, for each document in the portion of the documents in the set of document images:
retrieving a document image from the data storage;
processing the document image to identify text segments contained within the document image;
processing the text segments to identify subword embeddings associated with each of the text segments, wherein each subword embedding associated with a text segment represents a character group in the document image,
generating an n-dimensional vector for each text segment from its subword embeddings;
for each identified text segment, mapping one or more of the n-dimensional vectors to each of the identified text segments to generate for each identified text segment, a feature vector which describes a local context of the identified text segment;
retrieving an annotated version of the document image containing a visual indication annotation associated with each visual indication of a keyword in the document; and
associating with each visual indication of a keyword in the annotated version of the document image a corresponding feature vector to generate a training document for the set of training documents.

8. The document processing system of claim 7 wherein the subword embeddings utilize a vector representation of one or more n-character groupings of a word, where n is a preselected integer and where a word is represented by a sum of the vector representations.

9. The document processing system of claim 7 wherein the instructions that when executed cause, for each identified text segment, mapping one or more of the n-dimensional vectors to each of the identified text segments to generate for each identified text segment, a feature vector which describes a local context of the identified text segment, comprise instructions that when executed cause the processor to:
select for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment.

10. The document processing system of claim 7 wherein the instructions that when executed cause, for each identified text segment, mapping one or more of the n-dimensional vectors to each of the identified text segments to generate for each identified text segment, a feature vector which describes a local context of the identified text segment, comprise instructions that when executed cause the processor to:
select for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment and which overlap the identified text segment by greater than a preselected overlap amount.

11. The document processing system of claim 7 wherein the instructions that when executed cause, for each identified text segment, mapping one or more of the n-dimensional vectors to each of the identified text segments to generate for each identified text segment, a feature vector which describes a local context of the identified text segment, comprise instructions that when executed cause the processor to:
select for the identified text segment one or more of text segments that are positioned above, below, to the left and to the right of the identified text segment, and wherein the feature vector comprises a concatenation of vectors corresponding to the identified text segment and of a vector corresponding to each of the text segments that are positioned above, below, to the left and to the right of the identified text segment.

12. The document processing system of claim 7 further comprising instructions that when executed cause the processor to provide the set of training documents to a supervised learning engine to generate a trained model.

13. A computer program product for generating a set of training documents, the computer program product comprising a non-transitory computer readable storage medium and including instructions for causing the computer system to execute a method for generating a set of training documents, the method comprising the actions of:
retrieving a document image from data storage which has stored thereon a set o

Assignees

• Automation Anywhere Inc

Inventors

• Corcoran Thomas
• Gejji Vibhas
• Van Lare Stephen

Classifications

• G06F16/313 Primary

  Selection or weighting of terms for indexing · CPC title

• G06V10/23

  based on positionally close patterns or neighbourhood relationships · CPC title

• G06V30/416

  Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors · CPC title

• G06V30/414

  Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title