Out-of bounds detection of a document in a live camera feed

Patent metadata

Field	Value
Publication number	US-10659643-B2
Application number	US-201816191632-A
Country	US
Kind code	B2
Filing date	Nov 15, 2018
Priority date	Jun 14, 2017
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

Aspects of the present disclosure provide methods and apparatuses for processing a digital image of a document, for example, to determine whether the document is a long document. An exemplary method generally includes obtaining a plurality of digital images of the document, segmenting at least a first digital image of the plurality of images into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image, detecting a plurality of contours in the segmented first digital image, deciding, for each detected contour of the plurality of contours, whether that contour is an open contour or a closed contour, and determining that one or more sides of the document is out-of-bounds based, at least in part, on the decisions.

First claim

What is claimed is:

1. A computer-implemented method for processing digital images of a document, comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;
detecting a contour in the segmented first digital image;
deciding whether the contour is an open contour or a closed contour;
determining that a side of a plurality of sides of the document is out-of-bounds based on whether the contour is an open contour;
informing a user that the document is out-of-bounds;
determining which particular side of the plurality of sides of the document is out-of-bounds based on which side of a bounding rectangle of the contour touches an edge of the first digital image;
informing the user that the particular side of the plurality of sides of the document is out-of-bounds; and
directing the user to capture at least one additional image of the document.

2. The method of claim 1 , wherein determining that the side of the plurality of sides of the document is out-of-bounds comprises determining whether a side of the contour touches one or more edges of the segmented first digital image.

3. The method of claim 2 , wherein determining that the side of the plurality of sides of the document is out-of-bounds further comprises determining one or more corners of the contour that are out-of-bounds.

4. The method of claim 1 , wherein directing the user to capture the at least one additional image of the document comprises alerting the user to capture a single image that fully encompasses the document.

5. The method of claim 1 , wherein directing the user to capture the at least one additional image of the document comprises:
when a font size on the document is greater than or equal to an upper font size threshold, directing the user to capture a single image of the document at a further distance than the first digital image.

6. The method of claim 1 , wherein directing the user to capture the at least one additional image of the document comprises:
when a font size of text on the document is less than or equal to a lower font size threshold, directing the user to capture multiple images of the document, each image of the multiple images focusing on a different portion of the document, wherein a combination of the multiple images of the document entirely encompasses the document.

7. The method of claim 6 , further comprising stitching together the multiple images of the document.

8. The method of claim 1 , further comprising stopping processing of the first digital image based on the determining that the side of the plurality of sides of the document is out-of-bounds.

9. An apparatus for processing digital images of a document, comprising:
a processor; and
a memory having instructions which, when executed by the processor, performs an operation for processing a digital image, the operation comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;
detecting a contour in the segmented first digital image;
deciding whether the contour is an open contour or a closed contour;
determining that a side of a plurality of sides of the document is out-of-bounds based on whether the contour is an open contour;
informing a user that the document is out-of-bounds;
determining which particular side of the plurality of sides of the document is out-of-bounds based on which side of a bounding rectangle of the contour touches an edge of the first digital image;

informing the user that the particular side of the plurality of sides of the document is out-of-bounds; and
directing the user to capture at least one additional image of the document.

10. The apparatus of claim 9 , wherein determining that the side of the plurality of sides of the document is out-of-bounds comprises determining whether a side of the contour touches one or more edges of the segmented first digital image.

11. The apparatus of claim 10 wherein determining that the side of the plurality of sides of the document is out-of-bounds further comprises determining one or more corners of the contour that are out-of-bounds.

12. The apparatus of claim 9 , wherein directing the user to capture the at least one additional image of the document comprises alerting the user to capture a single image that fully encompasses the document.

13. The apparatus of claim 9 , wherein directing the user to capture the at least one additional image of the document comprises:
when a font size on the document is greater than or equal to an upper font size threshold, directing the user to capture a single image of the document at a further distance than the first digital image.

14. The apparatus of claim 9 , wherein directing the user to capture the at least one additional image of the document comprises:
when a font size of text on the document is less than or equal to a lower font size threshold, directing the user to capture multiple images of the document, each image of the multiple images focusing on a different portion of the document, wherein a combination of the multiple images of the document entirely encompasses the document.

15. The apparatus of claim 14 , wherein the operation further comprises stitching together the multiple images of the document.

16. The apparatus of claim 9 , further comprising stopping processing of the first digital image based on the determining that the side of the plurality of sides of the document is out-of-bounds.

17. A non-transitory computer-readable medium comprising instructions which, when executed on one or more processors, performs an operation for processing a digital image of a document, comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;
detecting a contour in the segmented first digital image;
deciding whether the contour is an open contour or a closed contour;
determining that a side of a plurality of sides of the document is out-of-bounds based on whether the contour is an open contour;
informing a user that the document is out-of-bounds;
determining which particular side of the plurality of sides of the document is out-of-bounds based on which side of a bounding rectangle of the contour touches an edge of the first digital image;
informing the user that the particular side of the plurality of sides of the document is out-of-bounds; and
directing the user to capture at least one additional image of the document.

18. The non-transitory computer-readable medium of claim 17 , wherein determining that the side of the plurality of sides of the document is out-of-bounds comprises determining whether a side of the contour touches one or more edges of the segmented first digital image.

19. The non-transitory computer-readable medium of claim 18 wherein determining that the side of the plurality of sides of the document is out-of-bounds further comprises determining one or more corners of the contour that are out-of-bounds.

20. The non-transitory computer-readable medium of claim 17 , wherein directing the user to capture the at least one additional image of the document comprises alerting the user to cap

Assignees

• Intuit Inc

Inventors

• Yellapragada Vijay S
• Chiang Peijun
• Lee Daniel
• Hall Jason
• Soliwal Shailesh

Classifications

• H04N1/00718

  Skew · CPC title

• H04N2201/0081

  Image reader (H04N2201/0091 - H04N2201/0094 take precedence) · CPC title

• H04N1/00771

  Indicating or reporting, e.g. issuing an alarm · CPC title

• H04N1/387

  Composing, repositioning or otherwise {geometrically} modifying originals · CPC title

• H04N1/00721

  Orientation · CPC title