Methods and devices for quantifying text similarity

Patent metadata

Field	Value
Publication number	US-11210553-B2
Application number	US-202117181839-A
Country	US
Kind code	B2
Filing date	Feb 22, 2021
Priority date	May 21, 2019
Publication date	Dec 28, 2021
Grant date	Dec 28, 2021

Abstract

Disclosed herein are computer-implemented methods; computer-implemented systems; and non-transitory, computer-readable media, for quantifying text similarity. One computer-implemented method includes obtaining a plurality of shortest operation paths including one or more edit pairs for correcting an optical correction recognition (OCR) text string with an edit text string, where each of the one or more edit pairs denotes an operation performable to a character of the OCR text string during correction by the edit text string. A plurality of similarity scores is determined, each corresponding to one of the plurality of shortest operation paths and determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths. A minimum one of the plurality of similarity scores is selected to quantify text similarity between the OCR text string and the edit text string.

First claim

What is claimed is:

1. A computer-implemented device for quantifying text similarity, comprising:
at least one processor; and
memory storing computer program code which when executed by the at least one processor, cause the at least one processor to:
obtain a plurality of shortest operation paths for correcting an optical correction recognition (OCR) text string with an edit text string, wherein each of the plurality of shortest operation paths includes one or more edit pairs, each of the one or more edit pairs denoting an operation performable to a character of the OCR text string during correction by the edit text string,
determine a plurality of similarity scores, each of the plurality of similarity scores corresponding to one of the plurality of shortest operation paths, wherein each of the plurality of similarity scores is determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths, wherein when summing the historical similarity scores of the one or more edit pairs, the computer program code further causes the at least one processor to:
retrieve the historical similarity scores of the one or more edit pairs from a history data library,
add the edit pairs in the shortest operation path having the minimum similarity score into the history data library,
update the historical similarity scores for the edit pairs in the history data library, wherein when updating the historical similarity scores, the computer program code further causes the at least one processor to:
calculate frequencies of edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score, and

determine historical similarity scores for the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score by: performing a log frequency calculation for each of the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score and normalizing the log frequency calculations to a range of 0.0 to 1.0, and

select a minimum one of the plurality of similarity scores to quantify text similarity between the OCR text string and the edit text string.

2. The computer-implemented device of claim 1 , wherein, when obtaining the plurality of shortest operation paths, the computer program code further causes the at least one processor to:
perform an edit distance calculation for correcting the OCR text string with the edit text string, wherein the operation performed on a character of the OCR text string during correction by the edit text string is one of an insertion operation, a deletion operation, or a substitution operation.

3. The computer-implemented device of claim 1 , wherein the computer program code further causes the at least one processor to:
determine that the minimum one of the plurality of similarity scores is below a predetermined threshold; and
in response to determining that the minimum one of the plurality of similarity scores is below a predetermined threshold, correct the OCR text string with the edit text string.

4. The computer-implemented device of claim 1 , wherein the computer program code further causes the at least one processor to:
determine that the minimum one of the plurality of similarity scores is above a predetermined threshold; and
in response to determining that the minimum one of the plurality of similarity scores is above the predetermined threshold, maintain the OCR text string.

5. The computer-implemented device of claim 1 , wherein the computer program code further causes the at least one processor to:
scan a digital image to capture the OCR text string, and
capture the edit text string.

6. A computer-implemented method for quantifying text similarity, comprising:
obtaining a plurality of shortest operation paths for correcting an optical correction recognition (OCR) text string with an edit text string, wherein each of the plurality of shortest operation paths includes one or more edit pairs, each of the one or more edit pairs denoting an operation performable to a character of the OCR text string during correction by the edit text string;
determining a plurality of similarity scores, each of the plurality of similarity scores corresponding to one of the plurality of shortest operation paths, wherein each of the plurality of similarity scores is determined by summing historical similarity scores of the one or more edit pairs of each of the plurality of shortest operation paths, wherein summing the historical similarity scores of the one or more edit pairs comprises:
retrieving the historical similarity scores of the one or more edit pairs from a history data library,
adding the edit pairs in the shortest operation path having the minimum similarity score into the history data library,
updating the historical similarity scores for the edit pairs in the history data library, wherein updating the historical similarity scores comprises:
calculating frequencies of edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score,

determining historical similarity scores for the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score by: performing a log frequency calculation for each of the edit pairs in the history data library corresponding to the edit pairs in the shortest operation path having the minimum similarity score and normalizing the log frequency calculations to a range of 0.0 to 1.0; and

selecting a minimum one of the plurality of similarity scores to quantify text similarity between the OCR text string and the edit text string.

7. The computer-implemented method of claim 6 , wherein the step of obtaining the plurality of shortest operation paths comprises performing an edit distance calculation for correcting the OCR text string with the edit text string, and wherein the operation performable to a character of the OCR text string during correction by the edit text string is one of an insertion operation, a deletion operation, or a substitution operation.

8. The computer-implemented method of claim 6 , further comprising:
determining that the minimum one of the similarity scores is below a predetermined threshold; and
in response to determining that the minimum one of the plurality of similarity scores is below a predetermined threshold, correcting the OCR text string with the edit text string.

9. The computer-implemented method of claim 8 , further comprising:
determining that the minimum one of the similarity scores is above a predetermined threshold; and
in response to determining that the minimum one of the similarity scores is above the predetermined threshold, maintaining the OCR text string.

10. The computer-implemented method of claim 6 , further comprising:
scanning a digital image to capture the OCR text string, and
capturing the edit text string.

11. A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform one or more operations to index blockchain data for storage, comprising:
obtaining a plurality of shortest operation paths for correcting an optical correction recognition (OCR) text string with an edit text string, wherein each of the plurality of shortest operation paths includes one or more edit pairs, each of the one or more edit pairs denoting an operation performable to a character of the OCR text string during correction by the edit te

Assignees

• Advanced New Technologies Co Ltd

Inventors

• Li Ruoyu

Classifications

• G06V30/1983 Primary

  Syntactic or structural pattern recognition, e.g. symbolic string recognition · CPC title

• G06V30/12

  Detection or correction of errors, e.g. by rescanning the pattern · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/22 Primary

  Matching criteria, e.g. proximity measures · CPC title

• G06V30/10

  Character recognition · CPC title