Methods and systems for adding annotations from a printed version of a document to a digital version of the document

Patent metadata

Field	Value
Publication number	US-12236185-B2
Application number	US-202217689246-A
Country	US
Kind code	B2
Filing date	Mar 8, 2022
Priority date	Mar 8, 2022
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

The present disclosure discloses methods and systems for adding one or more annotations from a printed version of a document to a digital version of the document. The methods and systems include receiving the printed document with one or more annotations, which represent review comments of a reviewer. The printed document including one or more annotations is scanned to obtain a scanned document. Thereafter, the scanned document is compared with the original digital version of the document to identify the one or more annotations. The identified one or more annotations are then extracted and added to the digital version of the document to obtain a new digital version, which can be used for changes by the user or any other user.

First claim

What is claimed is:

1. A method for adding annotations from a printed version of a document to a digital version of the document, the method comprising:
receiving the printed version of the document having one or more annotations from a user;
scanning the printed version of the document comprising the one or more annotations to generate a scanned digital version of the document;
for each of the one or more annotations, identifying first textual content of each of the one or more annotations from the scanned document using optical character recognition, wherein the first textual content comprises one or more characters;
for each of the one or more annotations, determining a calculated confidence value of accuracy of the identifying the first textual content; and
for each of the one or more annotations having a calculated confidence value above a predetermined threshold confidence value:
identifying a location to associate each of the one or more annotations in the digital version of the document, the identifying using a trained machine learning module to recognize the first textual content of each of the one or more annotations as corresponding to second textual content of the digital version of the document, wherein the second textual comprises one or more characters and differs from the first textual content; and
adding each of the one or more annotations to the digital version of the document at the identified location, wherein for one or more of the annotations having a calculated confidence value above the predetermined threshold confidence value, adding the annotation further comprises anchoring the annotation to an associated word, phrase, sentence, paragraph or page in the digital version of the document.

2. The method of claim 1 , wherein the one or more annotations are hand-written annotations.

3. The method of claim 1 , further comprising extracting the one or more annotations from the scanned document.

4. The method of claim 1 , further comprising embedding any of the one or more annotations with confidence values less than the predetermined threshold confidence value as image in the digital version of the document.

5. The method of claim 1 , further comprising:
allowing the user or another user to make changes in a new digital version of the document comprising the one or more added annotations; and
receiving changes from the user or another user made in the new digital version of the document based on the one or more added annotations.

6. The method of claim 5 , further comprising storing the changes made by the user in the new digital version of the document.

7. The method of claim 1 , wherein the machine learning module is trained using training data comprising previously generated annotations correlated to digital content.

8. A multi-function device for adding annotations from a printed version of a document to a digital version of the document, the multi-function device comprising:
a scanner for scanning a printed version of a document having one or more annotations;
a document manager for:
each of the one or more annotations, identifying first textual content of the one or more annotations from the scanned digital version of the document using optical character recognition, wherein the first textual content comprises one or more characters;
each of the one or more annotations, determining a calculated confidence value of accuracy of the identifying the first textual content; and
each of the one or more annotations having a calculated confidence value above a predetermined threshold confidence value:
identifying a location to associate each of the one or more annotations in the digital version of the document, the identifying using a trained machine learning module to recognize the first textual content of the one or more annotations as corresponding to second textual content of the digital version of the document, wherein the second textual content comprises one or more characters and differs from the first textual content; and
adding each of the one or more annotations to the digital version of the document at the identified location, wherein for one or more of the annotations having a calculated confidence value above the predetermined threshold confidence value, adding the annotation comprises anchoring the annotation to an associated word, phrase, sentence, paragraph or page in the digital version of the document.

9. The multi-function device of claim 8 , wherein the one or more annotations are hand-written annotations.

10. The multi-function device of claim 8 , wherein the document manager extracts the one or more annotations from the scanned document.

11. The multi-function device of claim 8 , wherein the document manager sends a new digital version of the document having the one or more annotations added to the user or to another user to make changes in the new digital version based on the one or more annotations.

12. The multi-function device of claim 11 , wherein the document manager stores the changes made by the user or the another user in the new digital version.

13. The multi-function device of claim 12 , wherein the document manager learns the stored changes made by the user for later identifying one or more other annotations in one or more upcoming scanned documents.

14. The multi-function device of claim 8 , further comprising a user interface for allowing a user to upload the digital version of the document for comparison.

15. The multi-function device of claim 8 , wherein the machine learning module is trained using training data comprising previously generated annotations correlated to digital content.

16. A non-transitory computer-readable medium storing instruction, which when executed by one or more processors cause the one or more processors to:
receiving a printed version of a document comprising one or more annotations from a user;
scanning the printed version of the document comprising the one or more annotations to generate a scanned document;
for each of the one or more annotations, identifying first textual content of the one or more annotations from the scanned document using optical character recognition, wherein the first textual content comprises one or more characters;
for each of the one or more annotations, determining a calculated confidence value of accuracy of the identifying the first textual content of each of the one or more annotations; and
for each of the one or more annotations having a calculated confidence value above a predetermined threshold confidence value:
identifying a location to associate each of the one or more annotations in the digital version of the document, the identifying using a trained machine learning module to recognize the first textual content of each of the one or more annotations as corresponding to second textual content of the digital version of the document, wherein the second textual comprises one or more characters and differs from the first textual content; and
adding each of the one or more annotations to the digital version of the document at the identified location, wherein for one or more of the annotations having a calculated confidence value above the predetermined threshold confidence value, adding the annotation comprises anchoring the annotation to an associated word, phrase, sentence, paragraph or page in the digital version of the document.

17. The non-transitory computer-readable medium storing instruction of claim 16 , wherein the one or more annotations are hand-written annotations.

18. The non-transitory computer-readable medium

Assignees

• Xerox Corp

Inventors

• Jayaraj Devadoss Shoban Kumar
• Gopalakrishnan Sainarayanan
• Kanagasabai Rajasekar

Classifications

• H04N1/32144 Primary

  embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp · CPC title

• G06F40/169 Primary

  Annotation, e.g. comment data or footnotes · CPC title