Method for identifying handwritten characters on an image using a classification model

Patent metadata

Field	Value
Publication number	US-12327422-B2
Application number	US-202217815706-A
Country	US
Kind code	B2
Filing date	Jul 28, 2022
Priority date	May 2, 2022
Publication date	Jun 10, 2025
Grant date	Jun 10, 2025

Abstract

A system for identifying handwritten characters on an image using a classification model that employs a neural network. The system includes a computer having a processor and a memory device that stores data and executable code that, when executed, causes the processor to read and convert typed text on the image to machine encoded text to identify locations of the typed text on the image; identify a location on the image that includes handwritten text based on the location of predetermined typed text on the image; identify clusters of non-white pixels in the image at the location having the handwritten text, where constraints are employed to refine and limit the clusters; generate an individual and separate cluster image for each identified cluster; and classify each cluster image using machine learning and at least one neural network to determine the likelihood that the cluster is a certain character.

First claim

What is claimed is:

1. A system for identifying handwritten characters on an image, said system comprising:
a computer including at least one processor and a memory device storing data and executable code that, when executed, causes the at least one processor to:
read and convert typed text on the image to machine encoded text to identify locations of the typed text on the image;
identify a location on the image that includes handwritten text based on the location of predetermined typed text on the image;
identify clusters of non-white pixels in the image at the location having the handwritten text, where each cluster is presumed to be a handwritten character, wherein the at least one processor employs a constraint that requires all of the clusters to be within a certain percentage size of each other;
generate an individual and separate cluster image for each identified cluster;
classify each cluster image using machine learning and at least one neural network to determine the likelihood that the cluster is a certain character; and
determine what character each cluster image is based on the classification.

2. The system according to claim 1 wherein the at least one processor reads and converts the typed text using an optical character recognition algorithm.

3. The system according to claim 1 wherein the at least one processor identifies clusters of non-white pixels using a density-based clustering algorithm.

4. The system according to claim 1 wherein the at least one processor rescales the cluster when it generates an individual and separate cluster image for each identified cluster.

5. The system according to claim 4 wherein the at least one processor rescales the cluster to a 28×28 pixel cluster image.

6. The system according to claim 1 wherein the at least one processor centers the cluster in the cluster image when it generates an individual and separate cluster image for each identified cluster.

7. The system according to claim 1 wherein the at least one processor classifies each cluster image by determining the likelihood that the cluster image is one of sixty-two character classes, namely, upper case letters A-Z, lower case letters a-z and numbers 0-9.

8. The system according to claim 1 wherein the at least one neural network is a convolutional neural network (CNN).

9. A system for identifying handwritten characters on an image of a check, said system comprising:
a computer including at least one processor and a memory device storing data and executable code that, when executed, causes the at least one processor to:
read and convert typed text on the image to machine encoded text using an optical character recognition algorithm to identify locations of the typed text on the image including identifying the location of the typed words “pay to the order of”;
identify a location on the image that includes a handwritten payee based on the location of the typed words “pay to the order of”;
identify clusters of non-white pixels in the image at the location having the handwritten payee using a density-based clustering algorithm, where each cluster is presumed to be a handwritten character, wherein the at least one processor employs a constraint that requires all of the clusters to be within a certain percentage size of each other;
generate an individual and separate cluster image for each identified cluster;
classify each cluster image using machine learning and at least one neural network to determine the likelihood that the cluster is a certain character, wherein the at least one processor classifies each cluster image by determining the likelihood that the cluster image is one of sixty-two character classes, namely, upper case letters A-Z, lower case letters a-z and numbers 0-9; and
determine what character each cluster image is based on the classification.

10. The system according to claim 9 wherein the at least one processor rescales the cluster when it generates an individual and separate cluster image for each identified cluster.

11. The system according to claim 10 wherein the at least one processor rescales the cluster to a 28×28 pixel cluster image.

12. The system according to claim 9 wherein the at least one processor centers the cluster in the cluster image when it generates an individual and separate cluster image for each identified cluster.

13. A method for identifying handwritten characters on an image, said method comprising:
reading and converting typed text on the image to machine encoded text to identify locations of the typed text on the image;
identifying a location on the image that includes handwritten text based on the location of predetermined typed text on the image;
identifying clusters of non-white pixels in the image at the location having the handwritten text, where each cluster is presumed to be a handwritten character, wherein identifying clusters of non-white pixels includes employing a constraint that requires all of the clusters to be within a certain percentage size of each other;
generating an individual and separate cluster image for each identified cluster,
classifying each cluster image using machine learning and at least one neural network to determine the likelihood that the cluster is a certain character; and
determining what character each cluster image is based on the classification.

14. The method according to claim 13 wherein identifying clusters of non-white pixels in the image includes employing a constraint that limits a size of each cluster to be less than a predetermined size.

15. The method according to claim 13 wherein generating an individual and separate cluster image for each identified cluster includes rescaling each cluster.

16. The method according to claim 15 wherein each cluster is rescaled to a 28×28 pixel cluster image.

17. The method according to claim 13 wherein generating an individual and separate cluster image for each identified cluster includes centering the cluster in the image.

18. The method according to claim 13 wherein classifying each cluster image includes determining the likelihood that the cluster image is one of sixty-two character classes, namely, upper case letters A-Z, lower case letters a-z and numbers 0-9.

19. The method according to claim 13 wherein the at least one neural network is a convolutional neural network (CNN).

20. The method according to claim 13 wherein the image is an image of a check, the predetermined typed text is “pay to the order of” and the handwritten text is a payee on the check.

Assignees

• Truist Bank

Inventors

• Fitzgerald Raphael

Classifications

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06V30/15

  Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques · CPC title

• G06V30/226 Primary

  of cursive writing · CPC title

• G06Q40/02

  Banking, e.g. interest calculation or account maintenance (credit or loans G06Q40/03) · CPC title

• G06N20/00

  Machine learning · CPC title