Method and system for accessing table content in a digital image of the table

Patent metadata

Field	Value
Publication number	US-11755347-B2
Application number	US-202217695188-A
Country	US
Kind code	B2
Filing date	Mar 15, 2022
Priority date	Jan 26, 2017
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

A system and method for accessing elements of a table in a digital image of the table, including: obtaining the digital image of the table; finding table elements in the digital image based on digital table properties, wherein the table elements define table cells; calculating coordinates of the table cells in the digital image based on the table elements; and accessing content of a selected table cell in the digital image using the coordinates of the selected table element.

First claim

The invention claimed is:

1. A computer implemented method for accessing elements of a table in a digital image-of the table, the method comprising:
obtaining the digital image of the table;
finding table elements in the digital image based on digital table properties by processing the digital image using imaging processing techniques, wherein the table elements define table cells;
calculating relative locations of the table cells in the digital image based on the table elements; and
accessing content of a selected table cell in the digital image using the relative location of the selected table element.

2. The method of claim 1 , wherein the table elements comprises at least one of gridlines and table text blocks.

3. The method of claim 1 , wherein the digital table properties comprise at least one of: has row separators, has column separators, has row header, has row header separators, has column header, has column header separators, table direction, single line, selection mode and a fragmentation method.

4. The method of claim 3 , comprising obtaining at least one of the digital table properties in a design phase prior to runtime.

5. The method of claim 3 , wherein finding table elements in the digital image is performed using a fragmentation method that is selected from the list consisting of: an orthogonality filter method, a gridlines detection method, a morphology detection method, a table skeleton method a cartoonizing method and Fast line detector method.

6. The method of claim 5 , wherein the orthogonality filter method comprises:
converting the digital image to a grayscale image;
applying a horizontal discrete differentiation operator to the grayscale image to calculate a horizontal derivative and vertical discrete differentiation operator to calculate a vertical derivative for pixels of the grayscale image, wherein the horizontal derivative and the vertical derivative of a pixel form a two-dimensional (2D) vector of the pixel;
calculating magnitudes and angles of the 2D vectors of the pixels of the grayscale image;
marking a pixel (i,j) as pertaining to a gridline if the pixel (i,j) and its (i+1,j), (i,j+1), (i+1,j+1) neighbors are orthogonal, wherein i and j are pixel indexes; and
extracting relative locations of vertical and horizontal gridlines.

7. The method of claim 6 , wherein the cartoonizing method comprises:
downsampling to the digital image to obtain a down sampled image;
performing edge detection on the down sampled image to obtain a processed image;
applying a bilateral filter to the processed image to obtain a filtered image;
upsampling the filtered image to obtain a zoomed-out image;
smoothing the zoomed-out image by applying a median filter with n×n aperture, wherein n is an odd number, to obtain a of smoothed image;
converting the smoothed image to a smoothed grayscale image;
converting the smoothed grayscale image to a binary image with adaptive threshold;
resizing the binary image to the image size of the digital image to obtain a resized binary image; and
applying the orthogonality filter method to the resized binary image.

8. The method of claim 5 , wherein the gridlines detection method comprises:
building grid mask by:
converting digital image of the table to a grayscale image;
blurring the grayscale image by applying a gaussian blur function to obtain a blurred image;
thresholding the blurred image to obtain a binary image;
scanning the binary image to obtain a pixel that is connected to a maximal number of pixels with a given color of the binary image; and
filling the connected pixels with a first color and the rest of the pixels with a second color to generate a mask; and

extracting relative locations of horizontal and vertical gridlines from the mask.

9. The method of claim 5 , wherein the morphology detection method comprises:
thresholding the digital image to obtain a binary image;
blurring the binary image to obtain a blurred image;
performing edge detection on the blurred image to obtain a processed image;
obtaining a structuring element;
performing morphological image processing on the processed image using the structuring element, to find geometry shapes;
identifying contours of the geometry shapes using a contour approximation method, wherein the contours represent text blocks in the table image; and
generating vertical and horizontal lines between the table blocks, wherein the vertical and horizontal lines form the gridlines of the digital image.

10. The method of claim 5 , wherein the table skeleton method comprises:
applying a Laplacian filter to the digital image to obtain a filtered image;
normalizing the filtered image to obtain a color image;
thresholding the color image to obtain a binary image;
blurring the binary image to obtain a blurred image;
calculating contours of a first color in the blurred image, wherein the contours represent text blocks in the digital image; and
generating vertical and horizontal lines between the table blocks, wherein the vertical and horizontal lines form the gridlines of the digital image.

11. The method of claim 1 , wherein the relative locations are relative to a point in the digital image, an anchor or a control.

12. The method of claim 1 , comprising presenting the digital image on a client device and wherein the computer is remote from the client device.

13. A system for accessing elements of a table in a digital image of a table, the system comprising:
a memory; and
a processor configured to:
obtain the digital image of the table;
find gridlines in the digital image based on digital table properties by processing the digital image using imaging processing techniques, wherein gridlines define table cells;
calculate relative locations of the table cells in the digital image based on the table elements; and
access content of a selected table cell in the digital image using the relative location of the selected table element.

14. The system of claim 13 , wherein the digital table properties comprise at least one of: has row separators, has column separators, has row header, has row header separators, has column header, has column header separators, table direction, single line, selection mode and a fragmentation method.

15. The system of claim 13 , wherein finding table elements in the digital image is performed using a fragmentation method that is selected from the list consisting of: an orthogonality filter method, a gridlines detection method, a morphology detection method, a table skeleton method a cartoonizing method and Fast line detector method.

16. The system of claim 15 , wherein the orthogonality filter method comprises:
converting the digital image to a grayscale image;
applying a horizontal discrete differentiation operator to the grayscale image to calculate a horizontal derivative and vertical discrete differentiation operator to calculate a vertical derivative for pixels of the grayscale image, wherein the horizontal derivative and the vertical derivative of a pixel form a two-dimensional (2D) vector of the pixel;
calculating magnitudes and angles of the 2D vectors of the pixels of the grayscale image;
marking a pixel (i,j) as pertaining to a gridline if the pixel (i,j) and its (i+1,j), (i,j+1), (i+1,j+1) neighbors are orthogonal, wherein i and j are pixel indexes; and
extracting relative locations of vertical and horizontal gridlines.

17. The system of claim 16 , wherein the cartoonizing method comprises:
downsampling to the digital image to obtain a down sampled image;
performing edge detection

Assignees

• Nice Ltd

Inventors

• Vaindiner Alexander
• Shelest Vitaly
• Kleinerman Semyon

Classifications

• G06V30/412 Primary

  Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables · CPC title

• G06F9/451 Primary

  Execution arrangements for user interfaces · CPC title

• G06V30/416

  Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors · CPC title

• G06F8/10

  Requirements analysis; Specification techniques · CPC title

• G06T5/20

  using local operators · CPC title