Detecting long documents in a live camera feed

Patent metadata

Field	Value
Publication number	US-10257375-B2
Application number	US-201715623008-A
Country	US
Kind code	B2
Filing date	Jun 14, 2017
Priority date	Jun 14, 2017
Publication date	Apr 9, 2019
Grant date	Apr 9, 2019

Abstract

Aspects of the present disclosure provide methods and apparatuses for processing a digital image of a document, for example, to determine whether the document is a long document. An exemplary method generally includes obtaining a plurality of digital images of the document, determining a type of the document, loading one or more pre-defined metrics associated with the document based on the determined type of the document, determining one or more characteristics of the document based on one or more analyses performed on the plurality of digital images of the document, comparing the one or more characteristics of the document with the one or more pre-defined metrics, and determining the document to be a long document based, at least in part, on the comparison.

First claim

What is claimed is:

1. A computer-implemented method for processing digital images of a document, comprising:
obtaining a first digital image of a document from a user;
determining a document type of the document in the first digital image based on a textual content of the document;
determining a font size of text in the document in the first digital image;
determining that at least one part of the document in the first digital image is out of bounds of the first digital image based on:
a bounding rectangle with a largest area corresponding to an open contour; and
the bounding rectangle with the largest area touching one or more edges of the first digital image;
comparing the font size of text in the document with a font size range of the determined document type;
determining, based on the comparison and the determination that the at least one part of the document is out of bounds of the first digital image, that the document in the first digital image is a long document;
generating, based on the determination that the document in the first digital image is a long document, an alert for the user to capture a set of digital images of the document, wherein each digital image of the set of digital images of the document is of a different portion of the document;
obtaining the set of digital images of the document from the user;
generating a second digital image of the document based on the obtained set of digital images; and
performing OCR on the second digital image of the document.

2. The method of claim 1 , wherein obtaining the set of digital images comprises obtaining the set of digital images via a live stream captured by a camera.

3. The method of claim 1 , wherein the comparison of the font size of the document with the font size range of the determined document type further comprises determining the font size of the text in the document is less than a lower bound of the font size range.

4. The method of claim 1 , wherein determining the at least one part of the document in the first digital image is out of bounds of the first digital image further comprises:
segmenting the first digital image into:
a first set of pixels associated with a foreground of the first digital image; and
a second set of pixels associated with a background of the first digital image;
detecting a set of contours in the segmented first digital image;
determining a set of bounding rectangles based on the set of contours;
determining an area of each bounding rectangle in the set of bounding rectangles;
determining the bounding rectangle of the set of bounding rectangles with the largest area; and
determining, for each contour of the set of contours, whether the contour is an open contour or a closed contour, wherein:
an open contour is a bounding rectangle with one or more sides outside of the segmented first digital image; and
a closed contour is a bounding rectangle with all four sides inside of the segmented first digital image.

5. The method of claim 1 , wherein the determining the document type of the document in the first digital image based on a textual content of the document further comprises:
performing OCR on the first digital image to determine the textual content; and
comparing the textual content to a set of text indicative of certain types of documents.

6. The method of claim 1 , where the determining a font size of text in the document in the first digital image further comprises:
drawing a bounding rectangle around each line of text in the document;
determining a number of text blocks based on a number of bounding rectangles;
determining an average height of the number of text blocks; and
determining an estimated text size of the text in the document based on the average height of the number of text blocks.

7. The method of claim 4 , wherein the method further comprises determining that the document in the first digital image is not a long document based on:
a determination the document is not out of bounds, wherein the determination the document is not out of bounds further comprises:
the bounding rectangle with the largest area corresponding to a closed contour; and
the bounding rectangle with the largest area does not touch an edge of the first digital image; and
a determination the font size of the text in the document is at a top or above the font size range.

8. An apparatus for processing digital images of a document, comprising:
a processor; and
a memory having instructions which, when executed by the processor, performs an operation for processing a digital image, the operation comprising:
obtaining a first digital image of a document from a user;
determining a document type of the document in the first digital image, based on a textual content of the document;
determining a font size of text in the document in the first digital image;
determining that at least one part of the document in the first digital image is out of bounds of the first digital image based on:
a bounding rectangle with a largest area corresponding to an open contour; and
the bounding rectangle with the largest area touching one or more edges of the first digital image;
comparing the font size of text in the document with a font size range of the determined document type;
determining, based on the comparison and the determination that the at least one part of the document is out of bounds of the first digital image, that the document in the first digital image is a long document;
generating, based on the determination that the document in the first digital image is a long document, an alert for the user to capture a set of digital images of the document, wherein each digital image of the set of digital images of the document is of a different portion of the document;
obtaining the set of digital images of the document from the user;
generating a second digital image of the document based on the obtained set of digital images; and
performing OCR on the second digital image of the document.

9. The apparatus of claim 8 , wherein the comparison of the font size of text in the document with the font size range of the determined document type further comprises determining the font size of the text in the document is less than a lower bound of the font size range.

10. The apparatus of claim 8 , wherein the operation for determining the at least one part of the document in the first digital image is out of bounds of the first digital image further comprises:
segmenting the first digital image into:
a first set of pixels associated with a foreground of the first digital image; and
a second set of pixels associated with a background of the first digital image;

detecting a set of contours in the segmented first digital image;
determining a set of bounding rectangles based on the set of contours;
determining an area of each bounding rectangle in the set of bounding rectangles;
determining the bounding rectangle of the set of bounding rectangles with the largest area; and
determining, for each contour of the set of contours, whether the contour is an open contour or a closed contour, wherein:
an open contour is a bounding rectangle with one or more sides outside of the segmented first digital image; and
a closed contour is a bounding rectangle with all four sides inside of the segmented first digital image.

11. The apparatus of claim 8 , wherein obtaining the set of digital images comprises obtaining the set of digital images via a live stream captured by a camera.

12. The apparatus of claim 8 , wherein the determining the document type of the document in the first digital image based on a textual content of the document further co

Assignees

• Intuit Inc

Inventors

• Yellapragada Vijay
• Chiang Peijun
• Lee Daniel
• Hall Jason
• Soliwal Shailesh

Classifications

• G06V30/141

  using multiple overlapping images; Image stitching · CPC title

• G06V30/1452

  based on positionally close symbols, e.g. amount sign or URL-specific characters · CPC title

• H04N1/19594

  using a television camera or a still video camera · CPC title

• H04N1/00713 Primary

  Length · CPC title

• H04N1/00737

  using the scanning elements as detectors · CPC title