Augmented reality document processing

Patent metadata

Field	Value
Publication number	US-11080547-B2
Application number	US-201816235337-A
Country	US
Kind code	B2
Filing date	Dec 28, 2018
Priority date	Dec 28, 2018
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

A method for processing an image of a physical document to enable digital searching of the physical document includes: obtaining a digital copy of the physical document; receiving an instruction to search for a point of interest in the physical document; imaging the physical document and determining that the point of interest is on a surface of the physical document; and projecting, using a projector and onto the surface of the physical document, a visual-aid component that physically overlaps with the point of interest to highlight the point of interest on the surface of the physical document.

First claim

What is claimed is:

1. A method for processing an image of a physical document to enable digital searching of the physical document, comprising:
obtaining a digital copy of the physical document;
receiving an instruction to search for a point of interest in the physical document;
imaging the physical document and determining that the point of interest is on a surface of the physical document; and
projecting, using a projector and onto the surface of the physical document, a visual-aid component that physically overlaps with the point of interest to highlight the point of interest on the surface of the physical document, wherein
the physical document is a page within a stack of physical documents,
the digital copy includes all pages in the stack, and the method further comprises:
imaging a top-most page of the stack and determining that the point of interest is not on the top-most page;
imaging, after removing the top-most page of the stack in response to determining that the point of interest is not on the top-most page, a next-top-most page of the stack under the removed top-most page to find the point of interest;
determining that the point of interest is on a surface of the next-top-most page; and projecting, using the projector and onto the surface of the next-top-most page, the visual aid component to highlight the point of interest on the surface of the next-top-most page.

2. The method of claim 1 , further comprising:
comparing, in response to determining that the point of interest is not on the top-most page, the top-most page with a page in the digital copy of the stack that comprises the point of interest; and outputting, based on the comparison, an instruction to remove pages from the stack to reach a page within the stack with the point of interest.

3. The method of claim 2 , wherein the instruction to remove pages from the stack is projected, by a projector and within a predetermined distance from the physical documents, onto a same surface that the physical documents are disposed on.

4. The method of claim 3 , wherein the instruction to remove pages from the stack to is continuously projected and updated until a determination is made that the point of interest is on a page within the stack that is now a new top-most page after the pages are removed.

5. The method of claim 1 , wherein: the highlighted point of interest is a first occurrence of the point of interest in the stack, and the method further comprises: receiving an instruction to search for a second occurrence of the point of interest in the stack; outputting, based on searching the digital copy of the stack for the second occurrence, an instruction to remove pages from the stack to reach a page within the stack with the second occurrence; imaging the stack after removing the pages from the stack to reach the page within the stack with the second occurrence and determining that the second occurrence is on a surface of a new top-most page of the stack after the pages have been removed; and projecting, using the projector and onto the surface of the new top-most page, the visual aid component to highlight the second occurrence on the surface of the new top-most page.

6. The method of claim 1 , wherein obtaining the digital copy of the physical document further comprises: imaging and digitizing each page of the stack; storing the imaged and digitized stack in a memory as the digital copy of the physical documents; imaging the top-most page of the stack; comparing the top-most page of the stack to pages of the digital copy of the stack; determining, based on the comparison, that the top-most page of the stack matches one of the pages of the digital copy of the stack; and retrieving, based on the determination, the digital copy of the stack from the memory.

7. The method of claim 1 , wherein the instruction to search for the point of interest in the physical document is received as a voice command from a user.

8. The method of claim 1 , wherein the physical documents are imaged using a Projection with Interactive Capture (PIC) device that comprises an imager and the projector, and an optical character recognition and non-text content analysis is used to search the digital copy.

9. The method of claim 1 , wherein the point of interest is one selected from a group consisting of: a text character, a non-text character, a string of text, an image, a graph, and a table.

10. The CRM of claim 1 , wherein the computer readable program code further causes the computer to:
compare, in response to determining that the point of interest is not on the top-most page, top-most page with a page in the digital copy of the stack that comprises the point of interest; and output, based on the comparison, an instruction to remove pages from the stack to reach a page within the stack with the point of interest.

11. A non-transitory computer readable medium (CRM) storing computer readable program code for processing an image of a physical document to enable digital searching of the physical document embodied therein, the computer readable program code causes a computer to:
obtain a digital copy of the physical document; receive an instruction to search for a point of interest in the physical document;
image the physical document and determine that the point of interest is on a surface of the physical document; and
project, using a projector and onto the surface of the physical document, a visual-aid component that physically overlaps with the point of interest to highlight the point of interest on the surface of the physical document, wherein
the physical document is a page within a stack of physical documents, the digital copy includes all pages in the stack, and the computer readable program code further causes the computer to:
image a top-most page of the stack and determining that the point of interest is not on the top-most page;
image, after removing the top-most page of the stack in response to determining that the point of interest is not on the top-most page, a next-top-most page of the stack under the removed top-most page to find the point of interest;
determine that the point of interest is on a surface of the next-top-most page; and project, using the projector and onto the surface of the next-top-most page, the visual aid component to highlight the point of interest on the surface of the next-top-most page.

12. The CRM of claim 11 , wherein the highlighted point of interest is a first occurrence of the point of interest in the stack, and the computer readable program code further causes the computer to: receive an instruction to search for a second occurrence of the point of interest in the stack; output, based on searching the digital copy of the stack for the second occurrence, an instruction to remove pages from the stack to reach a page within the stack with the second occurrence; image the stack after removing the pages from the stack to reach the page within the stack with the second occurrence and determining that the second occurrence is on a surface of a new top-most page of the stack after the pages have been removed; and project, using the projector and onto the surface of the new top-most page, the visual aid component to highlight the second occurrence on the surface of the new top-most page.

13. The CRM of claim 11 , wherein the physical documents are imaged using a Projection with Interactive Capture (PIC) device that comprises an imager and the projector, an optical character recognition and non-text content analysis is used to search the digital copy, and the point of interest is one selected from a group consisting of: a text character, a non-text character, a string of te

Assignees

• Konica Minolta Laboratory Usa Inc

Inventors

• Tecu Kirk Steven

Classifications

• G06F16/5846 Primary

  using extracted text · CPC title

• G06F16/93

  Document management systems · CPC title

• G06K2209/01

  Physics · mapped topic

• G06K9/46 Primary

  Physics · mapped topic