Information processing with iteratively improved estimates of data attributes based on user modifications, and apparatus, method, and storage medium thereof

Patent metadata

Field	Value
Publication number	US-12148234-B2
Application number	US-202117552352-A
Country	US
Kind code	B2
Filing date	Dec 15, 2021
Priority date	Dec 24, 2020
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

Processing of an image obtained by scanning an original document and on which character recognition processing is executed on text blocks which are extracted from the scanned image. The text blocks indicate regions of character attributes which may have different data attributes. An estimate is made of the likelihood that a text block is associated with a predetermined data attribute, and a display is made of the scanned image and the character string included in the estimated text block. If the displayed character string is modified by a user, reference information is updated based on the modified character string and history information pertaining to the modification is displayed. The updated reference information is used to improve the estimate of the likelihood that the text block is associated with the predetermined data attribute.

First claim

What is claimed is:

1. An information processing system, comprising:
at least one memory that stores instructions; and
at least one processor that execute the instructions to:
obtain a scanned image obtained by scanning an original document;
execute character recognition processing on text blocks which are extracted from the scanned image, the text blocks indicating regions of character attributes;
estimate a text block including a character string associated with a predetermined data attribute out of the text blocks;
control a display device to display the scanned image and the character string included in the estimated text block; and
update, if the displayed character string is modified by a user, reference information based on the modified character string and display history information which indicates that a text block including the modified character string has displayed on the display device while the displayed character string is modified by the user, wherein the updated reference information is used such that the text block including the modified character string is to be estimated as a text block including a character string associated with the predetermined data attribute.

2. The information processing system according to claim 1 , wherein
the at least one processor estimates the text block including the character string associated with the predetermined data attribute by using a learning model, wherein the updated reference information include character string feature amount of the modified character string and the display history information, and wherein the learning model performs learning based on the updated reference information.

3. The information processing system according to claim 1 , wherein
the at least one processor identifies a registered image in which positions of text blocks are similar to that in the scanned image out of a plurality of registered images that each indicate a position of a text block including a character string associated with the predetermined data attribute, and estimates the text block including the character string associated with the predetermined data attribute based on the similar registered image, wherein the plurality of registered images are updated based on the updated reference information.

4. The information processing system according to claim 1 , wherein
if there are a plurality of text blocks including a character string matching the modified character string, the at least one processor selects one from the plurality of text blocks based on the display history information.

5. The information processing system according to claim 1 , wherein
the predetermined data attribute includes at least one of a title of the original document, an identification number of the original document, an issuer of the original document, a phone number of the issuer, an issue date of the original document, sub total, and total.

6. An information processing method, comprising:
obtaining a scanned image obtained by scanning an original document;
executing character recognition processing on text blocks which are extracted from the scanned image, the text blocks indicating regions of character attributes;
estimating a text block including a character string associated with a predetermined data attribute out of the text blocks;
controlling a display device to display the scanned image and the character string included in the estimated text block; and
updating, if the displayed character string is modified by a user, reference information based on the modified character string and display history information which indicates that a text block including the modified character string has displayed on the display device while the displayed character string is modified by the user, wherein the updated reference information is used such that the text block including the modified character string is to be estimated as a text block including a character string associated with the predetermined data attribute.

7. A non-transitory computer-readable storage medium storing a program to cause a computer to execute an information processing method, comprising:
obtaining a scanned image obtained by scanning an original document;
executing character recognition processing on the extracted text blocks which are extracted from the scanned image, the text blocks indicating regions of character attributes;
estimating a text block including a character string associated with a predetermined data attribute out of the text blocks;
controlling a display device to display the scanned image and the character string included in the estimated text block; and
updating, if the displayed character string is modified by a user, reference information based on the modified character string and display history information which indicates that a text block including the modified character string has displayed on the display device while the displayed character string is modified by the user, wherein the updated reference information is used such that the text block including the modified character string is to be estimated as a text block including a character string associated with the predetermined data attribute.

Assignees

• Canon Kk

Inventors

• Kosaka Ryo

Classifications

• G06V30/153 Primary

  using recognition of characters or words · CPC title

• G06V30/412

  Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables · CPC title

• G06V30/414 Primary

  Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text · CPC title