Systems and methods for detection and high-quality capture of documents on a cluttered tabletop with an automatically controlled camera

Patent metadata

Field	Value
Publication number	US-2016259971-A1
Application number	US-201514637391-A
Country	US
Kind code	A1
Filing date	Mar 3, 2015
Priority date	Mar 3, 2015
Publication date	Sep 8, 2016
Grant date	—

Abstract

Described are systems and methods for recognizing paper documents on a tabletop using an overhead camera mounted on pan-tilt servos. The described automated system first finds paper documents on a cluttered desk based on a text probability map, constructed using multiple images acquired at fixed grid positions, and then captures a sequence of high-resolution overlapping frames of the located document(s), which are then fused together and perspective-rectified, using computed homography, to reconstruct a high quality and fronto-parallel document image that is of sufficient quality required for optical character recognition. The extracted textual information may be used, for example, for indexing and search, document repository and/or language translation applications.

First claim

What is claimed is:

1 . A computer-implemented method being performed in a computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, the computer-implemented method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface; b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface; c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and d. extracting the text using the second captured plurality of images.

2 . The computer-implemented method of claim 1 , wherein the processing unit is configured to cause the turret to move the camera during the scanning and capturing.

3 . The computer-implemented method of claim 1 , wherein in a. the first plurality of images is acquired by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the first plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was acquired.

4 . The computer-implemented method of claim 3 , wherein a. further comprises, for each image in the first plurality of images of the surface, computing feature points and a text response map.

5 . The computer-implemented method of claim 4 , wherein the each text response map is computed using a probability histogram, the probability histogram being pre-computed based on the feature points.

6 . The computer-implemented method of claim 4 , wherein a. further comprises stitching the text response maps corresponding to the first plurality of images into a single text response map based on the plurality of fixed positions corresponding to the first plurality of images.

7 . The computer-implemented method of claim 6 , wherein a. further comprises detecting text blobs in the single text response map.

8 . The computer-implemented method of claim 7 , wherein a. further comprises identifying at least one large rectangular shaped blob from the detected text blobs as the location of the text on the surface.

9 . The computer-implemented method of claim 1 , wherein in b. the second plurality of images is captured by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the second plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was captured.

10 . The computer-implemented method of claim 9 , wherein the determined location of the text on the surface is a location of a text blob and wherein the predetermined path is a center line through a bounding box of the corresponding text blob.

11 . The computer-implemented method of claim 9 , wherein the images of the second plurality of images overlap with one another.

12 . The computer-implemented method of claim 11 , further comprising stitching and fusing the images of the second plurality of images to obtain a second stitched image.

13 . The computer-implemented method of claim 12 , further comprising fitting lines around a boundary of a text blob in the second stitched image and estimating vanishing points.

14 . The computer-implemented method of claim 12 , further comprising performing a perspective rectification on the second stitched image based on a computed homography.

15 . The computer-implemented method of claim 14 , wherein in d. the text is extracted using an optical character recognition performed on the perspective rectified second stitched image.

16 . A non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in a computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, cause the computerized system to perform a method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface; b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface; c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and d. extracting the text using the second captured plurality of images.

17 . The non-transitory computer-readable medium of claim 16 , wherein the set of computer-executable instructions configures the processing unit to cause the turret to move the camera during the scanning and capturing.

18 . The non-transitory computer-readable medium of claim 16 , wherein in a. the first plurality of images is acquired by moving the camera to a plurality of fixed positions along a predetermined path, wherein each image of the first plurality of images corresponds to a fixed position of the plurality of fixed positions, where the image was acquired.

19 . The non-transitory computer-readable medium of claim 16 , wherein a. further comprises, for each image in the first plurality of images of the surface, computing feature points and a text response map.

20 . A computerized system comprising a processing unit, a memory and a camera, the camera being mounted on a turret operatively coupled to the processing unit, the memory storing a set of computer-executable instructions causing the computerized system to perform a method comprising:
a. scanning a surface with the camera to acquire a first plurality of images of the surface; b. using the acquired first plurality of images of the surface to determine at least one location of a text on the surface; c. capturing a second plurality of images using the camera based on the determined location of the text on the surface; and d. extracting the text using the second captured plurality of images.

Assignees

• Fuji Xerox Co Ltd

Inventors

• Kim Chelhwon
• Chiu Patrick
• Tang Hao

Classifications

• G06V30/40

  Document-oriented image-based pattern recognition · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T7/004

  Physics · mapped topic