Adaptive enhancement of scanned document pages

Patent metadata

Field	Value
Publication number	US-10587773-B2
Application number	US-201816046888-A
Country	US
Kind code	B2
Filing date	Jul 26, 2018
Priority date	Dec 17, 2014
Publication date	Mar 10, 2020
Grant date	Mar 10, 2020

Abstract

Enhancing image quality of an initial full image of a document page includes obtaining the initial full image of a document page, determining that at least a subset of text in the initial full image does not meet a predefined readability criterion, identifying at least one of a plurality of page fragments in the initial full image of the document page for enhancement, and sending an instruction to a mobile device to provide a photograph of the at least one of the page fragments. The photograph provides a separate fragment image for the at least one of the page fragments. The separate fragment image is then obtained from the mobile device and merged into the initial full image to provide an enhanced full image.

First claim

What is claimed is:

1. A method implemented at a content management system for enhancing image quality of an initial full image of a document, comprising:
obtaining the initial full image of a document page, wherein the initial full image includes a plurality of predefined page fragments;
determining that at least a subset of text in the initial full image does not meet a predefined readability criterion;
identifying at least one of the plurality of predefined page fragments corresponding to the subset of text in the initial full image of the document page for enhancement;
generating an instruction for a mobile device to provide a photograph of the at least one of the predefined page fragments, wherein the photograph provides a separate fragment image for the at least one of the predefined page fragments; and
in response to the instruction, obtaining the separate fragment image provided by the mobile device, and merging the separate fragment image into the initial full image to provide an enhanced full image.

2. The method of claim 1 , wherein in accordance with the predefined readability criterion, a size of the subset of text in the initial full image is not smaller than a user-suggested text readability threshold.

3. The method of claim 1 , further comprising:
detecting a page border of the document page in the initial full image;
retrieving the document page from the initial full image; and
correcting the retrieved document page.

4. The method of claim 3 , wherein correcting the retrieved document page further includes at least one of: perspective correction, light correction, color correction, shape correction, contrast adjustment, noise removal, and dewarping of the retrieved document page.

5. The method of claim 3 , further comprising:
splitting the corrected document page into one or more of text lines, paragraphs and drawings.

6. The method of claim 3 , further comprising:
estimating the size of the at least a subset of text in the initial full image as viewed by a user in accordance with a plurality of predetermined algorithms.

7. The method of claim 1 , wherein the mobile device is configured to in response to the instruction, direct a user of the mobile device to capture the photograph, identify the photograph in its memory or receive the photograph from a distinct device.

8. The method of claim 1 , wherein the separate fragment image includes a first separate fragment image, further comprising:
identifying a second page fragment of the plurality of predefined page fragments in the initial full image of the document page for enhancement, wherein the second page fragment at least partially overlaps with the at least one of the predefined page fragments;
obtaining a second separate fragment image for the second page fragment; and
merging the second separate fragment image into the initial full image, including eliminating an overlap of the first and second separate fragment images.

9. The method of claim 1 , further comprising:
prior to identifying the at least one of the plurality of predefined page fragments in the initial full image, subdividing the initial full image of the document page into a predetermined number of segments, wherein each of the plurality of predefined page fragments includes at least one of the predetermined number of segments.

10. The method of claim 1 , further comprising:
determining that the subset of text in the initial full image does not meet the predefined readability criterion, including:
determining that the initial full image is displayed in a full page;
calculating a size of the subset of text in the initial full image displayed in the full page; and
comparing the size of the subset of text in the initial full image displayed in the full page with a readability threshold.

11. A computer system configured to host a content management system, comprising:
one or more processors; and
memory storing one or more programs to be executed by the one or more processors, the one or more programs comprising instructions for:
obtaining an initial full image of a document page, wherein the initial full image includes a plurality of predefined page fragments;
determining that at least a subset of text in the initial full image does not meet a predefined readability criterion;
identifying at least one of the plurality of predefined page fragments corresponding to the subset of text in the initial full image of the document page for enhancement;
generating an instruction for a mobile device to provide a photograph of the at least one of the predefined page fragments, wherein the photograph provides a separate fragment image for the at least one of the predefined page fragments; and
in response to the instruction, obtaining the separate fragment image provided by the mobile device, and merging the separate fragment image into the initial full image to provide an enhanced full image.

12. The computer system of claim 11 , wherein the one or more programs further comprise instructions for:
enabling display of a visual indicator for indicating a next one of the plurality of predefined page fragments to enhance and a navigation path across a subset of non-captured page fragments, wherein the next one of the predefined page fragments follows the at least one of the predefined page fragments on the navigation path and is recommended for being enhanced using a second separate fragment image.

13. The computer system of claim 11 , wherein the document page is one of: a page from a book, a page from a magazine, a printed newspaper article, a receipt, an invoice, a check, a tax form or other form, a printed report, one or more business cards, a handwritten note, a memo on a legal pad, a page from a notebook application, a sticky note application, and an easel.

14. The computer system of claim 11 , wherein the one or more programs further comprise instructions for:
determining a quality of the separate fragment image for the at least one of the page fragments; and
in accordance with a determination that the quality of the separate fragment image is substantially low, automatically deleting the separate fragment image.

15. The computer system of claim 11 , wherein the one or more programs further comprise instructions for:
providing an option to delete the separate fragment image in response to detecting an obstruction in the separate fragment image.

16. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system that is configured to host a content management system, the one or more programs comprising instructions for:
obtaining an initial full image of a document page, wherein the initial full image includes a plurality of predefined page fragments;
determining that at least a subset of text in the initial full image does not meet a predefined readability criterion;
identifying at least one of the plurality of predefined page fragments corresponding to the subset of text in the initial full image of the document page for enhancement;
generating an instruction for a mobile device to provide a photograph of the at least one of the predefined page fragments, wherein the photograph provides a separate fragment image for the at least one of the predefined page fragments; and
in response to the instruction, obtaining the separate fragment image provided by the mobile device, and merging the separate fragment image into the initial full image to provide an enhanced full image.

17. The non-transitory computer readable storage medium of claim 16 , wherein

Assignees

• Evernote Corp

Inventors

• Pashintsev Alexander
• Gorbatov Boris
• Kitainik Leonid

Classifications

• H04N1/0443

  Varying the scanning velocity or position · CPC title

• H04N1/3876 Primary

  Recombination of partial images to recreate the original image · CPC title

• H04N1/00442

  Simultaneous viewing of a plurality of images, e.g. using a mosaic display arrangement of thumbnails · CPC title

• H04N1/19594

  using a television camera or a still video camera · CPC title

• H04N2201/0084

  Digital still camera · CPC title