Verification of optical character recognition results

Patent metadata

Field	Value
Publication number	US-10068155-B2
Application number	US-201615275990-A
Country	US
Kind code	B2
Filing date	Sep 26, 2016
Priority date	Sep 16, 2016
Publication date	Sep 4, 2018
Grant date	Sep 4, 2018

Abstract

A method of verifying optical character recognition (OCR) results may involve: performing OCR on one or more initial images of a document and displaying initial OCR results of the document to a user; receiving a feedback from the user regarding an error location in the initial OCR results, the error location being a location of a misspelled character sequence; receiving an additional image of the document, which corresponds to the error location, and performing OCR of the additional image to produce additional OCR results; identifying a cluster of character sequences, which correspond to the error location, using the initial OCR results and the additional OCR results; identifying an order of character sequences in the cluster of character sequences based on their respective probability values; and displaying to the user modified optical character recognition results, which contain in the error location a corrected character sequence.

First claim

What is claimed is:

1. A method comprising:
performing optical character recognition on one or more initial images of a document to produce initial optical character recognition results and displaying the initial optical character recognition results of the document to a user;
receiving a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;
receiving an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;
performing optical character recognition of the additional image to produce additional optical character recognition results;
identifying a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;
performing, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;
identifying, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and
displaying to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

2. The method of claim 1 , wherein the additional image differs from the one or more initial images in at least one of: an image noise, an image scale, a shooting angle and an image brightness.

3. The method of claim 1 , wherein said identifying the cluster of character sequences, which correspond to the error location, comprises: identifying in the initial optical character recognition results and the additional optical character recognition results a plurality of common features to identify reference points.

4. The method of claim 3 , further comprising identifying using coordinates of the reference points parameters of a coordinate transformation converting coordinates of the additional optical character recognition results into coordinates of the initial optical character recognition results.

5. The method of claim 1 , wherein said identifying, based on the probability values, the character sequence of the cluster of character sequences as the corrected character sequence comprises determining a pre-determined metric for each character sequence of the cluster and resorting the character sequences of the cluster according to the probability values.

6. The method of claim 5 , wherein the pre-determined metric is a sum of edit distances between a character sequence of the cluster and each of the other character sequences of the cluster and wherein the character sequences are resorted so that a character sequence with a lowest value of the metric goes is the top of the cluster, while a character sequence with a highest value of the metric is on the bottom of the cluster.

7. The method of claim 1 , wherein the one or more initial images and the additional image are obtained in a continuous sequence of images.

8. The method of claim 1 , wherein displaying to user the modified optical character recognition results comprises highlighting in the displayed modified optical character recognition results the corrected character sequence.

9. The method of claim 1 , further comprising, after said receiving the feedback from the user, highlighting the error location in the displayed initial optical character recognition results.

10. The method of claim 1 , which performed by a mobile device.

11. The method of claim 10 , wherein the mobile device is one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

12. The method of claim 10 , wherein said receiving the feedback from the user comprises receiving a feedback from a location of a touch screen display of the mobile device, which corresponds to the error location.

13. The method of claim 10 , wherein the mobile device comprises a camera and the one or more initial images and the additional image are acquired by said camera.

14. A system comprising:
a memory;
a display; and
a processing device, which is coupled to the memory and the display, the processing device is configured to:
perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results to a user;
receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;
receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds the error location;
perform optical character recognition on the additional image to produce additional optical character recognition results;
identify a cluster of character sequences corresponding to the error location by matching the initial optical character recognition results and the additional optical character recognition results;
perform, for each of the cluster of characters, a probability evaluation to determine a plurality of probability values for the cluster of characters;
identify, based on the probability values, a character sequence of the cluster of character sequences as a corrected character sequence; and
display to the user modified optical character recognition results, which contain in the error location the corrected character sequence, wherein the corrected character sequence is different from the misspelled character sequence.

15. The system of claim 14 , which is a mobile device.

16. The system of claim 15 , wherein the mobile device is a one of a mobile phone, a tablet, a laptop, a smartphone or a PDA.

17. The system of claim 14 , wherein the display is a touch-screen display and wherein the processing device receives the feedback from the user regarding the error location by receiving a feedback from a location of the touch-screen display touched by the user.

18. The system of claim 14 , further comprising a camera configured to acquire the one or more images and the next image and to transfer each of the one or more images and the next image to the processing device.

19. A computer-readable non-transitory storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:
perform optical character recognition on one or more initial images of a document to produce initial optical character recognition results and display the initial optical character recognition results of the document to a user;
receive a feedback from the user regarding an error location in the initial optical character recognition results, wherein the error location is a location of a misspelled character sequence in the initial optical character recognition results;
receive an additional image of the document, wherein the additional image contains a portion of the document, which corresponds to the error location;
perform optical character recognition on the additional image to produce additional optical character recognition results;
identify a cluster of character sequences corresponding to the error location by matching the initial optical character re

Assignees

• Abbyy Dev Llc

Inventors

• Kalyuzhny Aleksey Ivanovich

Classifications

• G06V30/12 Primary

  Detection or correction of errors, e.g. by rescanning the pattern · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F40/20

  Natural language analysis (semantic analysis of natural language G06F40/30) · CPC title