Processing irregularly arranged characters

Patent metadata

Field	Value
Publication number	US-11574456-B2
Application number	US-201916594127-A
Country	US
Kind code	B2
Filing date	Oct 7, 2019
Priority date	Oct 7, 2019
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

Aspects of the present disclosure relate to processing irregularly arranged characters. An image is received. An irregularly arranged character within the image is detected. A direction of the irregularly arranged character is modified to a proper direction to obtain a properly oriented character. The properly oriented character is recognized to obtain a first identified character. The image is then rebuilt by replacing the irregularly arranged character with the first identified character, the first identified character in a machine-encoded format.

First claim

What is claimed is:

1. A method comprising:
receiving, by a processor, an image;
detecting a first irregularly arranged character within the image;
determining a first set of pixel coordinates of the first irregularly arranged character within the image;
modifying a direction of the first irregularly arranged character to a proper direction to obtain a first properly oriented character;
recognizing, using a neural network, the first properly oriented character to obtain a first identified character;
rebuilding the image by replacing the first irregularly arranged character with the first identified character at the determined first set of pixel coordinates at a first time, the first identified character in a machine-encoded format;
detecting a second irregularly arranged character within the image;
determining a second set of pixel coordinates of the second irregularly arranged character within the image;
modifying a direction of the second irregularly arranged character to a proper direction to obtain a second properly oriented character;
recognizing, using the neural network, the second properly oriented character to obtain a second identified character; and
rebuilding the image by replacing the second irregularly arranged character with the second identified character at the determined second set of pixel coordinates at a second time, the second identified character in the machine-encoded format.

2. The method of claim 1 , wherein the direction of the first or second irregularly arranged character is modified using a morphological feature-based model.

3. The method of claim 1 , wherein the first or second irregularly arranged character is detected using a region-based convolutional neural network.

4. The method of claim 1 , wherein the properly oriented character is recognized using a dense convolutional network.

5. The method of claim 1 , wherein prior to modifying the direction of the first or second irregularly arranged character, the irregularly arranged character is extracted from the image.

6. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:
receiving an image;
detecting a first irregularly arranged character within the image;
determining a first set of pixel coordinates of the first irregularly arranged character within the image;
modifying a direction of the first irregularly arranged character to a proper direction to obtain a first properly oriented character;
recognizing, using a neural network, the first properly oriented character to obtain a first identified character;
rebuilding the image by replacing the first irregularly arranged character with the first identified character at the determined first set of pixel coordinates at a first time, the first identified character in a machine-encoded format;
detecting a second irregularly arranged character within the image;
determining a second set of pixel coordinates of the second irregularly arranged character within the image;
modifying a direction of the second irregularly arranged character to a proper direction to obtain a second properly oriented character;
recognizing, using the neural network, the second properly oriented character to obtain a second identified character; and
rebuilding the image by replacing the second irregularly arranged character with the second identified character at the determined second set of pixel coordinates at a second time, the second identified character in the machine-encoded format.

7. The computer program product of claim 6 , wherein the direction of the first or second irregularly arranged character is modified using a morphological feature-based model.

8. The computer program product of claim 6 , wherein the first or second irregularly arranged character is detected using a region-based convolutional neural network.

9. The computer program product of claim 6 , wherein the properly oriented character is recognized using a dense convolutional network.

10. The computer program product of claim 6 , wherein prior to modifying the direction of the first or second irregularly arranged character, the first or second irregularly arranged character is extracted from the image.

11. A system comprising:
a memory storing program instructions; and
a processor, wherein the processor is configured to execute the program instructions to perform a method comprising:
receiving an image;
detecting a first irregularly arranged character within the image;
determining a first set of pixel coordinates of the first irregularly arranged character within the image;
modifying a direction of the first irregularly arranged character to a proper direction to obtain a first properly oriented character;
recognizing, using a neural network, the first properly oriented character to obtain a first identified character;
rebuilding the image by replacing the first irregularly arranged character with the first identified character at the determined first set of pixel coordinates at a first time, the first identified character in a machine-encoded format;
detecting a second irregularly arranged character within the image;
determining a second set of pixel coordinates of the second irregularly arranged character within the image;
modifying a direction of the second irregularly arranged character to a proper direction to obtain a second properly oriented character;
recognizing, using the neural network, the second properly oriented character to obtain a second identified character; and
rebuilding the image by replacing the second irregularly arranged character with the second identified character at the determined second set of pixel coordinates at a second time, the second identified character in the machine-encoded format.

12. The system of claim 11 , wherein the first or second irregularly arranged character is detected using a region-based convolutional neural network.

13. The system of claim 11 , wherein prior to modifying the direction of the first or second irregularly arranged character, the irregularly arranged character is extracted from the image.

14. The system of claim 11 , wherein prior to rebuilding the image by replacing the first or second irregularly arranged character with the first identified character, pixels associated with the pixel coordinates of the irregularly arrange character are altered to erase the first or second irregularly arranged character.

Assignees

• IBM

Inventors

• Cai Zhuo
• Yin Jian Dong
• Wang Wen
• Fu Rong
• Sheng Hao
• Zhang Kang

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06V10/242 Primary

  by image rotation, e.g. by 90 degrees · CPC title

• G06N3/045

  Combinations of networks · CPC title