Methods and devices for quantifying text similarity

Patent metadata

Field	Value
Publication number	US-2020372293-A1
Application number	US-202016791858-A
Country	US
Kind code	A1
Filing date	Feb 14, 2020
Priority date	May 21, 2019
Publication date	Nov 26, 2020
Grant date	—

Abstract

Disclosed herein are computer-implemented methods; computer-implemented systems; and non-transitory, computer-readable media, for quantifying text similarity. One computer-implemented method includes obtaining a plurality of shortest operation paths including one or more edit pairs for correcting an optical correction recognition (OCR) text string with an edit text string, where each of the one or more edit pairs denotes an operation performable to a character of the OCR text string during correction by the edit text string. A plurality of similarity scores is determined, each corresponding to one of the plurality of shortest operation paths and determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths. A minimum one of the plurality of similarity scores is selected to quantify text similarity between the OCR text string and the edit text string.

First claim

1 . A computer-implemented device for quantifying text similarity, the device comprising:
at least one processor; and a memory including computer program code for execution by the at least one processor, the computer program code configured to cause the device at least one processor to:
obtain a plurality of shortest operation paths for correcting an optical correction recognition (OCR) text string with an edit text string, wherein each of the plurality of shortest operation paths includes one or more edit pairs, each of the one or more edit pairs denoting an operation performable to a character of the OCR text string during correction by the edit text string;
determine a plurality of similarity scores, each of the plurality of similarity scores corresponding to one of the plurality of shortest operation paths, wherein each of the plurality of similarity scores is determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths, wherein when summing the historical similarity scores of the one or more edit pairs, the computer program codes further instruct the device to:
retrieve the historical similarity scores of the one or more edit pairs from a history data library;

collect edit pairs into the history data library;
calculate frequencies of the edit pairs in the history data library; and
determine the historical similarity scores for the edit pairs in the history data library, wherein when determining the historical similarity scores, the computer program codes further instruct the device to:
perform a log frequency calculation for each of the edit pairs in the history data library; and
normalize the log frequency calculations to a range of 0.0 to 1.0; and
select a minimum one of the plurality of similarity scores to quantify text similarity between the OCR text string and the edit text string.

2 - 3 . (canceled)

4 . The computer-implemented device of claim 1 , wherein the computer program code is further configured to cause the at least one processor to:
add the edit pairs in the shortest operation path having the minimum similarity score into the history data library; and update the historical similarity scores for the edit pairs in the history data library, wherein when updating the historical similarity scores, the computer-implemented device is caused to:
calculate frequencies of edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score; and
determine historical similarity scores for the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score by: performing a log frequency calculation for each of the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score and normalizing the log frequency calculations to a range of 0.0 to 1.0.

5 . The computer-implemented device of claim 1 , wherein, when obtaining the plurality of shortest operation paths, the computer program code is further configured to cause the at least one processor to:
perform an Edit Distance calculation for correcting the OCR text string with the edit text string, wherein the operation performable to a character of the OCR text string during correction by the edit text string is one of an insertion operation, a deletion operation, or a substitution operation.

6 . The computer-implemented device of claim 1 , wherein the computer program code is further configured to cause the at least one processor to:
if the minimum one of the plurality of similarity scores is below a predetermined threshold, correct the OCR text string with the edit text string.

7 . The computer-implemented device of claim 6 , wherein the computer program code is further configured to cause the at least one processor to:
if the minimum one of the plurality of similarity scores is above the predetermined threshold, maintain the OCR text string.

8 . The computer-implemented device of claim 1 , wherein the computer program code is further configured to cause the at least one processor to:
scan a digital image to capture the OCR text string, and capture the edit text string.

9 . A computer-implemented method for quantifying text similarity, comprising:
obtaining a plurality of shortest operation paths for correcting an optical correction recognition (OCR) text string with an edit text string, wherein each of the plurality of shortest operation paths includes one or more edit pairs, each of the one or more edit pairs denoting an operation performable to a character of the OCR text string during correction by the edit text string; determining a plurality of similarity scores for the plurality of shortest operation paths, wherein each of the plurality of similarity scores is determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths, wherein summing the historical similarity scores of the one or more edit pairs comprises retrieving the historical similarity scores of the one or more edit pairs from a history data library; collecting edit pairs into the history data library; calculating frequencies of the edit pairs in the history data library; and determining the historical similarity scores for the edit pairs in the history data library, wherein calculating the historical similarity scores comprises:
performing a log frequency calculation for each of the edit pairs in the history data library; and
normalizing the log frequency calculations to a range of 0.0 to 1.0; and
selecting a minimum one of the plurality of similarity scores to quantify text similarity between the OCR text string and the edit text string.

10 - 11 . (canceled)

12 . The computer-implemented method of claim 9 , further comprising:
adding the edit pairs in the shortest operation path having the minimum similarity score into the history data library; and updating the historical similarity scores for the edit pairs in the history data library, wherein updating the historical similarity scores comprises:
calculating frequencies of edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score; and
determining historical similarity scores for the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score by: performing a log frequency calculation for each of the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score and normalizing the log frequency calculations to a range of 0.0 to 1.0.

13 . The computer-implemented method of claim 9 , wherein the step of obtaining the plurality of shortest operation paths comprises performing an Edit Distance calculation for correcting the OCR text string with the edit text string, and wherein the operation performable to a character of the OCR text string during correction by the edit text string is one of an insertion operation, a deletion operation, or a substitution operation.

14 . The computer-implemented method of claim 9 , further comprising:
if the minimum one of the similarity scores is below a predetermined threshold, correcting the OCR text string with the edit text string.

15 . The computer-implemented method of claim 14 , further comprising:
if the minimum one of the similarity scores is above the predetermined threshold, maintaining the OCR text

Assignees

• Alibaba Group Holding Ltd

Inventors

• Li Ruoyu

Classifications

• G06V30/1983 Primary

  Syntactic or structural pattern recognition, e.g. symbolic string recognition · CPC title

• G06V30/12

  Detection or correction of errors, e.g. by rescanning the pattern · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22 Primary

  Matching criteria, e.g. proximity measures · CPC title

• G06V30/10

  Character recognition · CPC title