Automating creation of accurate OCR training data using specialized UI application

Patent metadata

Field	Value
Publication number	US-10289905-B2
Application number	US-201816112190-A
Country	US
Kind code	B2
Filing date	Aug 24, 2018
Priority date	Oct 5, 2016
Publication date	May 14, 2019
Grant date	May 14, 2019

Abstract

Systems of the present disclosure generate accurate training data for optical character recognition (OCR). Systems disclosed herein generates images of a text passage as displayed piecemeal in a user interface (UI) element rendered in a selected font type and size, determine accurate dimensions and locations of bounding boxes for each character pictured in the images, stitch together a training image by concatenating the images, and associate the training image, the bounding box dimensions and locations, and the text passage together in a collection of training data. The collection of training data also includes a computer-readable master copy of the text passage with newline characters inserted therein.

First claim

What is claimed is:

1. A method for generating training data for optical character recognition (OCR), the method comprising:
receiving a training data passage comprising a plurality of characters;
for each respective font type of a plurality of font types:
for each respective font size of a plurality of font sizes:
for each respective character in the plurality of characters:
displaying the respective character alone in a character UI element in the respective font type and in the respective font size;
capturing an image of the respective character as displayed in the character UI element;
determining dimensions of the image of the respective character; and
storing the dimensions of the image in a data structure, wherein the dimensions stored in the data structure are associated with the respective character.

2. The method of claim 1 , further comprising: for each respective character in the training data passage:
determining a set of offsets for the respective character based on the image of the respective character, wherein each offset of the set of offsets indicates a distance between an edge of the image of the respective character and the respective character; and
storing the set of offsets in the data structure, wherein the set of offsets stored in the data structure are associated with the respective character.

3. The method of claim 2 , further comprising: for each respective character in the training data passage: storing the respective font type and the respective font size in the data structure, wherein the respective font type and the respective font size stored in the data structure are associated with the respective character.

4. The method of claim 3 , further comprising:
receiving a request for a width of a bounding box associated with a selected character of the plurality of characters; and
providing the width of the bounding box based on dimensions of an image stored in the data structure associated with the selected character.

5. The method of claim 4 , wherein the width of the bounding box is measured in pixels.

6. The method of claim 3 , further comprising:
receiving a request for a set of offsets associated with a selected character of the plurality of characters; and
providing the set of offsets stored in the data structure associated with the selected character.

7. The method of claim 1 , further comprising:
receiving a user selection of the plurality of font types; and
receiving a user selection of the plurality of font types.

8. A system for generating training data for optical character recognition (OCR), comprising:
one or more processors; and
memory storing computer-executable instructions that, when executed on the one or more processors of the system, perform an operation for generating training data for optical character recognition (OCR), the operation comprising:
receiving a training data passage comprising a plurality of characters;
for each respective font type of a plurality of font types:
for each respective font size of a plurality of font sizes:
for each respective character in the plurality of characters:
 displaying the respective character alone in a character UI element in the respective font type and in the respective font size;
 capturing an image of the respective character as displayed in the character UI element;
 determining dimensions of the image of the respective character; and
 storing the dimensions of the image in a data structure, wherein the dimensions stored in the data structure are associated with the respective character.

9. The system of claim 8 , wherein the operation further comprises: for each respective character in the training data passage:
determining a set of offsets for the respective character based on the image of the respective character, wherein each offset of the set of offsets indicates a distance between an edge of the image of the respective character and the respective character; and
storing the set of offsets in the data structure, wherein the set of offsets stored in the data structure are associated with the respective character.

10. The system of claim 9 , wherein the operation further comprises: for each respective character in the training data passage: storing the respective font type and the respective font size in the data structure, wherein the respective font type and the respective font size stored in the data structure are associated with the respective character.

11. The system of claim 10 , wherein the operation further comprises:
receiving a request for a width of a bounding box associated with a selected character of the plurality of characters; and
providing the width of the bounding box based on dimensions of an image stored in the data structure associated with the selected character.

12. The system of claim 11 , wherein the width of the bounding box is measured in pixels.

13. The system of claim 10 , wherein the operation further comprises:
receiving a request for a set of offsets associated with a selected character of the plurality of characters; and
providing the set of offsets stored in the data structure associated with the selected character.

14. The system of claim 8 , wherein the operation further comprises:
receiving a user selection of the plurality of font types; and
receiving a user selection of the plurality of font types.

15. A non-transitory computer-readable storage medium comprising instructions for performing a method of generating training data for optical character recognition (OCR), the method comprising:
receiving a training data passage comprising a plurality of characters;
for each respective font type of a plurality of font types:
for each respective font size of a plurality of font sizes:
for each respective character in the plurality of characters:
displaying the respective character alone in a character UI element in the respective font type and in the respective font size;
capturing an image of the respective character as displayed in the character UI element;
determining dimensions of the image of the respective character; and
storing the dimensions of the image in a data structure, wherein the dimensions stored in the data structure are associated with the respective character.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the method further comprises: for each respective character in the training data passage:
determining a set of offsets for the respective character based on the image of the respective character, wherein each offset of the set of offsets indicates a distance between an edge of the image of the respective character and the respective character; and
storing the set of offsets in the data structure, wherein the set of offsets stored in the data structure are associated with the respective character.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises: for each respective character in the training data passage: storing the respective font type and the respective font size in the data structure, wherein the respective font type and the respective font size stored in the data structure are associated with the respective character.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the method further comprises:
receiving a request for a width of a bounding box associated with a selected character of the plurality of characters; and
providing the width of the bounding box based on dimensions of an image stored in the data s

Assignees

• Intuit Inc

Inventors

• Krivopaltsev Eugene
• Maddika Sreeneel K
• Yellapragada Vijay S

Classifications

• G06V30/1914 Primary

  Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries, e.g. user dictionaries · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/28

  Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title