Detecting orientation of textual documents on a live camera feed

Patent metadata

Field	Value
Publication number	US-10163007-B2
Application number	US-201715499666-A
Country	US
Kind code	B2
Filing date	Apr 27, 2017
Priority date	Apr 27, 2017
Publication date	Dec 25, 2018
Grant date	Dec 25, 2018

Abstract

The present disclosure relates to the extraction of text from an image including a depiction of a document. According to one embodiment, a mobile device receives an image depicting a document. The mobile device identifies a plurality of text areas in the document and identifies a midpoint of each of the plurality of text areas in the document. The mobile device detects one or more lines of text in the document including a plurality of text areas, where the plurality of text areas included in a line of text are associated with a midpoint having a coordinate within a threshold number of pixels on one axis in a two-dimensional space. Based on an orientation of the detected one or more lines of text, the mobile device determines a probable orientation of the document and extracts text from the image based on the determined probable orientation of the document.

First claim

What is claimed is:

1. A method for performing optical character recognition on an image of a document, comprising:
receiving the image of the document;
identifying a plurality of text areas in the document, wherein each of the plurality of text areas corresponds to a continuous set of non-whitespace characters detected in the document;
identifying a midpoint of each of the plurality of text areas in the document wherein the identified midpoint of each of the plurality of text areas has a pixel coordinate location in a two-dimensional space;
detecting one or more lines of text in the document, wherein each of the one or more lines of text comprises a set of text areas having midpoints with a vertical pixel coordinate located within a threshold number of pixels on a vertical axis in the two-dimensional space or a horizontal pixel coordinate located within a threshold number of pixels on a horizontal axis in two-dimensional space;
determining a probable orientation of the document based on an orientation of the detected one or more lines of text, wherein the probable orientation indicates that lines of text in the document are oriented parallel with a vertical axis or a horizontal axis; and
extracting text from the image based on the determined probable orientation of the document.

2. The method of claim 1 , wherein the probable orientation of the document comprises a first rotation angle and a second rotation angle, and wherein the second rotation angle differs from the first rotation angle by 180 degrees.

3. The method of claim 2 , wherein extracting text from the image comprises:
generating a first image by rotating the received image by the first rotation angle;
extracting text from the first image; and
upon determining that the extracted text from the first image comprises valid text, returning the extracted text to an application for processing.

4. The method of claim 3 , further comprising:
upon identifying a failure to extract valid text from the first image:
generating a second image by rotating the received image by the second rotation angle; and
extracting text from the second image.

5. The method of claim 4 , wherein identifying a failure to extract valid text from the first image comprises determining that a number of strings extracted from the first image do not match strings in a dictionary of known words.

6. The method of claim 1 , wherein determining the probable orientation of the document comprises determining that the image is mirrored about a vertical axis if the detected one or more lines of text include a plurality of text areas with a similar midpoint on a vertical axis of the image.

7. The method of claim 1 , wherein determining the probable orientation of the document comprises determining that the image is mirrored about a horizontal axis if the detected one or more lines of text include a plurality of text areas with a similar midpoint on a horizontal axis of the image.

8. A non-transitory computer-readable medium comprising instructions which, when executed on a processor, performs an operation for performing optical character recognition on an image of a document, the operation comprising:
receiving the image of the document;
identifying a plurality of text areas in the document wherein each text area corresponds to a continuous set of non-whitespace characters detected in the document;
identifying a midpoint of each of the plurality of text areas in the document, wherein the identified midpoint of each of the plurality of text areas has a pixel coordinate location in a two-dimensional space;
detecting one or more lines of text in the document, wherein each of the one or more lines of text comprises a set of text areas, wherein the set of text areas have midpoints with a vertical pixel coordinate located within a threshold number of pixels on a vertical axis in the two-dimensional space or a horizontal pixel coordinate located within a threshold number of pixels on a horizontal axis in the two-dimensional space, and wherein each of the plurality of text areas is associated with a midpoint having a coordinate within a threshold number of pixels on one axis in a two-dimensional space;
determining a probable orientation of the document based on an orientation of the detected one or more lines of text, wherein the probable orientation indicates that lines of text in the document are oriented parallel with a vertical axis or a horizontal axis; and
extracting text from the image based on the determined probable orientation of the document.

9. The non-transitory computer-readable medium of claim 8 , wherein the probable orientation of the document comprises a first rotation angle and a second rotation angle, and wherein the second rotation angle differs from the first rotation angle by 180 degrees.

10. The non-transitory computer-readable medium of claim 9 , wherein extracting text from the image comprises:
generating a first image by rotating the received image by the first rotation angle;
extracting text from the first image; and
upon determining that the extracted text from the first image comprises valid text, returning the extracted text to an application for processing.

11. The non-transitory computer-readable medium of claim 10 , wherein the operation further comprises:
upon identifying a failure to extract valid text from the first image:
generating a second image by rotating the received image by the second rotation angle; and
extracting text from the second image.

12. The non-transitory computer-readable medium of claim 11 , wherein identifying a failure to extract valid text from the first image comprises determining that a number of strings extracted from the first image do not match strings in a dictionary of known words.

13. The non-transitory computer-readable medium of claim 8 , wherein determining the probable orientation of the document comprises one of:
determining that the image is mirrored about a vertical axis if the detected one or more lines of text include a plurality of text areas with a similar midpoint on a vertical axis of the image; or
determining that the image is mirrored about a horizontal axis if the detected one or more lines of text include a plurality of text areas with a similar midpoint on a horizontal axis of the image.

14. A system, comprising:
a processor; and
a memory comprising instructions which, when executed on the processor, performs an operation for performing an operation for performing optical character recognition on an image of document, the operation comprising:
receiving the image of the document;
identifying a plurality of text areas in the document wherein each text area corresponds to a continuous set of non-whitespace characters detected in the document;
identifying a midpoint of each of the plurality of text areas in the document, wherein the identified midpoint of each of the plurality of text areas has a pixel coordinate location in a two-dimensional space;

detecting one or more lines of text in the document, wherein each of the one or more lines of text comprises a set of text areas, wherein the set of text areas have midpoints with a vertical pixel coordinate located within a threshold number of pixels on a vertical axis in the two-dimensional space or a horizontal pixel coordinate located within a threshold number of pixels on a horizontal axis in two-dimensional space;
determining a probable orientation of the document based on an orientation of the detected one or more lines of text, wherein the probable orientation indicates that lines of text in the document are oriented parallel with a vertical axis or a ho

Assignees

• Intuit Inc

Inventors

• Lee Daniel
• Yellapragada Vijay
• Soliwal Shailesh
• Chiang Peijun

Classifications

• G06V30/1463 Primary

  Orientation detection or correction, e.g. rotation of multiples of 90 degrees · CPC title

• G06V30/414

  Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text · CPC title

• G06V30/268

  Lexical context · CPC title

• G06V30/10

  Character recognition · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title