Live document detection in a captured video stream

Patent metadata

Field	Value
Publication number	US-12475300-B2
Application number	US-202418744349-A
Country	US
Kind code	B2
Filing date	Jun 14, 2024
Priority date	Jul 22, 2016
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

The present disclosure is directed toward systems and methods to quickly and accurately identify boundaries of a displayed document in a live camera image feed, and provide a document boundary indicator within the live camera image feed. For example, systems and methods described herein utilize different display document detection processes in parallel to generate and provide a document boundary indicator that accurately corresponds with a displayed document within a live camera image feed. Thus, a user of the mobile computing device can easily see whether the document identification system has correctly identified the displayed document within the camera viewfinder feed.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
detecting a visual representation of a physical document within a live camera feed from a camera on a mobile device; providing, for display within a graphical user interface of the mobile device, a live boundary indicator associated with the visual representation of the physical document within the live camera feed, the live boundary indicator updating based on changes to a position or an orientation of the visual representation of the physical document within the live camera feed; capturing an image of the visual representation of the physical document from the live camera feed; converting the image of the visual representation of the physical document that is in a picture file format to a document file that is in a document file format; and sending the document file to a second computing device for storage.

2 . The computer-implemented method of claim 1 , wherein sending the document file to the second computing device for storage comprises sending the document file to a content management system causing the content management system to send the document file to the second computing device.

3 . The computer-implemented method of claim 1 , wherein the mobile device and the second computing device are both associated with a user account.

4 . The computer-implemented method of claim 1 , wherein converting the image of the visual representation of the physical document comprises:
generating an enhanced image from the image of the visual representation of the physical document; and converting the enhanced image of the visual representation of the physical document that is in a picture file format to a document file that is in a document file format the image of the visual representation of the physical document is in a picture file format; and converting the image of the visual representation of the physical document to a document file comprises converting the image from the picture file format to a document file format.

5 . The computer-implemented method of claim 1 , wherein the mobile device is associated with a first user account and the second computing device is associated with a second user account.

6 . The computer-implemented method as recited in claim 1 , further comprising providing a shaded overlay comprising a document area within a perimeter of the live boundary indicator and a non-document area outside the perimeter of the live boundary indicator, wherein the document area has a different shading compared to the non-document area.

7 . The computer-implemented method as recited in claim 1 , further comprising:
cropping the image of the visual representation of the physical document based on the live boundary indicator; and providing a preview of the cropped image prior to converting the image of the visual representation of the physical document to the document file.

8 . A non-transitory computer-readable medium storing instructions thereon that, when executed by at least one processor, cause a mobile device to:
detect a visual representation of a physical document within a live camera feed from a camera on the mobile device, the mobile device associated with a user account; provide, for display within a graphical user interface of the mobile device, a live boundary indicator associated with the visual representation of the physical document within the live camera feed; capture an image of the visual representation of the physical document from the live camera feed; generate an enhanced image from the image of the visual representation of the physical document; convert the enhanced image of the visual representation of the physical document that is in a picture file format to a document file that is in a document file format; and provide the document file to a second client device associated with the user account.

9 . The non-transitory computer-readable medium as recited in claim 8 , wherein providing the document file to the second client device associated with the user account comprises providing the document file to a content management system that synchronizes the document file on the second client device.

10 . The non-transitory computer-readable medium as recited in claim 8 , wherein generating the enhanced image from the image of the visual representation of the physical document comprises cropping the image based on the live boundary indicator.

11 . The non-transitory computer-readable medium as recited in claim 8 , wherein:
the image of the visual representation of the physical document is in a picture file format; and converting the image of the visual representation of the physical document to a document file comprises converting the image from the picture file format to a document file format.

12 . The non-transitory computer-readable medium as recited in claim 8 , wherein providing the document file to the second client device associated with the user account is in response to receiving a user selection of a share file option.

13 . The non-transitory computer-readable medium as recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the mobile device to provide a preview of the enhanced image prior to converting the enhanced image to the document file.

14 . The non-transitory computer-readable medium as recited in claim 8 , further comprising instructions that, when executed by the at least one processor, cause the mobile device to provide a shaded overlay comprising a document area within a perimeter of the live boundary indicator and a non-document area outside the perimeter of the live boundary indicator, wherein the document area has a different shading compared to the non-document area.

15 . A system comprising:
at least one processor; and at least one non-transitory computer-readable storage medium storing instructions thereon that, when executed by the at least one processor, cause the system to:
detect a visual representation of a physical document within a live camera feed from a camera on a mobile device, the mobile device associated with a user account;
provide, for display within a graphical user interface of the mobile device, a live boundary indicator associated with the visual representation of the physical document within the live camera feed;
capture an image of the visual representation of the physical document from the live camera feed;
convert the image of the visual representation of the physical document that is in a picture file format to a document file that is in a document file format; and
provide the document file to a second client device associated with the user account.

16 . The system as recited in claim 15 , wherein providing the document file to the second client device associated with the user account comprises providing the document file to a content management system that synchronizes the document file on the second client device.

17 . The system as recited in claim 15 , further comprising instructions that, when executed by the at least one processor, cause the system to:
crop the image of the visual representation of the physical document based on the live boundary indicator; and provide a preview of the cropped image prior to converting the image of the visual representation of the physical document to the document file.

18 . The system as recited in claim 15 , further comprising instructions that, when executed by the at least one processor, cause the system to provide a shaded overlay comprising a document

Assignees

• Dropbox Inc

Inventors

• Welinder Nils Peter
• Belhumeur Peter N
• Xiong Ying
• Baek Jongmin
• Kozlov Simon
• Berg Thomas
• Kriegman David J

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T5/90

  Dynamic range modification of images or parts thereof · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/10

  Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) · CPC title