Out-of-bounds detection for a document in a live camera feed

Patent metadata

Field	Value
Publication number	US-11140290-B2
Application number	US-202016850530-A
Country	US
Kind code	B2
Filing date	Apr 16, 2020
Priority date	Jun 14, 2017
Publication date	Oct 5, 2021
Grant date	Oct 5, 2021

Abstract

Aspects of the present disclosure provide methods and apparatuses for processing a digital image of a document, for example, to determine whether the document is a long document. An exemplary method generally includes obtaining a plurality of digital images of the document, segmenting at least a first digital image of the plurality of images into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image, detecting a plurality of contours in the segmented first digital image, deciding, for each detected contour of the plurality of contours, whether that contour is an open contour or a closed contour, and determining that one or more sides of the document is out of bounds based, at least in part, on the decisions.

First claim

What is claimed is:

1. A method for processing digital images of a document, comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;
detecting a contour in the segmented first digital image;
determining whether the contour is an open contour or a closed contour;
determining whether a bounding rectangle with a largest area a plurality of bounding rectangles bounds the contour;
determining that a side of a plurality of sides of the document is out of bounds based on whether the contour is an open contour and based on whether the bounding rectangle with the largest area of the plurality of bounding rectangles bounds the contour;
determining which particular side of the plurality of sides of the document is out of bounds based on which side of the bounding rectangle with the largest area of the plurality of bounding rectangles touches an edge of the first digital image; and
generating a notification that the particular side of the plurality of sides of the document is out of bounds.

2. The method of claim 1 , wherein determining that the side of the plurality of sides of the document is out of bounds comprises determining whether the side of the contour touches one or more edges of the first digital image.

3. The method of claim 2 , wherein determining that the side of the plurality of sides of the document is out of bounds further comprises determining one or more corners of the contour that are out of bounds.

4. The method of claim 1 , wherein the notification includes an indication to capture at least one additional image of the document.

5. The method of claim 1 , wherein, when a font size on the document is greater than or equal to an upper font size threshold, the notification includes an indication to capture a single image of the document at a further distance than the first digital image.

6. The method of claim 1 , wherein, when a font size of text on the document is less than or equal to a lower font size threshold, the notification includes an indication to capture multiple images of the document, each image of the multiple images focusing on a different portion of the document, wherein a combination of the multiple images of the document entirely encompasses the document.

7. The method of claim 6 , further comprising:
receiving the multiple images of the document; and
stitching together the multiple images of the document.

8. The method of claim 1 , further comprising stopping processing of the first digital image based on the determining that the side of the plurality of sides of the document is out of bounds.

9. A system, comprising:
a processor; and
a memory having instructions that, when executed by the processor, cause the system to perform a method for processing a digital image, the method comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;

detecting a contour in the segmented first digital image;
determining whether the contour is an open contour or a closed contour;
determining whether a bounding rectangle with a largest area of a plurality of bounding rectangles bounds the contour;
determining that a side of a plurality of sides of the document is out of bounds based on whether the contour is an open contour and based on whether the bounding rectangle will the largest area of the plurality of bounding rectangles bounds the contour;
determining which particular side of the plurality of sides of the document is out of bounds based on which side of the bounding rectangle with the largest area of the plurality of bounding rectangles touches an edge of the first digital image; and
generating a notification that the particular side of the plurality of sides of the document is out of bounds.

10. The system of claim 9 , wherein determining that the side of the plurality of sides of the document is out of bounds comprises determining whether the side of the contour touches one or more edges of the first digital image.

11. The system of claim 10 , wherein determining that the side of the plurality of sides of the document is out of bounds further comprises determining one or more corners of the contour that are out of bounds.

12. The system of claim 9 , wherein the notification includes an indication to capture at least one additional image of the document.

13. The system of claim 9 , wherein, when a font size on the document is greater than or equal to an upper font size threshold, the notification includes an indication to capture a single image of the document at a further distance than the first digital image.

14. The system of claim 9 , wherein, when a font size of text on the document is less than or equal to a lower font size threshold, the notification directs a user to capture multiple images of the document, each image of the multiple images focusing on a different portion of the document, wherein a combination of the multiple images of the document entirely encompasses the document.

15. The system of claim 14 , wherein the method further comprises:
receiving the multiple images of the document; and
stitching together the multiple images of the document.

16. The system of claim 9 , wherein the method further comprises stopping processing of the first digital image based on the determining that the side of the plurality of sides of the document is out of bounds.

17. A computer-implemented method for processing digital images of a document, comprising:
segmenting a first digital image of a document into pixels associated with a foreground of the first digital image and pixels associated with a background of the first digital image to produce a segmented first digital image;
identifying a rectangle in the segmented first digital image;
determining whether bounding a rectangle with a largest area of a plurality of bounding rectangles bounds the rectangle;
determining that a side of a plurality of sides of the document is out of bounds based on whether the side of the rectangle touches an edge of the first digital image and based on whether the bounding rectangle with the largest area of the plurality of bounding rectangles bounds the rectangle;
determining which particular side of the plurality of sides of the document is out of bounds based on which side of the bounding rectangle with the largest area of the plurality of bounding rectangles touches the edge of the first digital image; and
generating a notification that the particular side of the plurality of sides of the document is out of bounds.

18. The method of claim 17 , wherein determining that the side of the plurality of sides of the document is out of bounds comprises determining one or more corners of the rectangle that are out of bounds.

19. The method of claim 17 , wherein the notification includes an indication to capture at least one additional image of the document.

20. The method of claim 17 , wherein, when a font size on the document is greater than or equal to an upper font size threshold, the notification includes an indication to capture a single image of the document at a further distance than the first digital image.

Assignees

• Intuit Inc

Inventors

• Yellapragada Vijay S
• Chiang Peijun
• Lee Daniel
• Hall Jason
• Soliwal Shailesh

Classifications

• H04N1/00748 Primary

  Detecting edges, e.g. of a stationary sheet · CPC title

• H04N1/00713 Primary

  Length · CPC title

• G06V30/142

  using hand-held instruments; Constructional details of the instruments · CPC title

• G06V20/63

  Scene text, e.g. street names · CPC title

• G06V30/40

  Document-oriented image-based pattern recognition · CPC title