Character recognition of license plate under complex background

Patent metadata

Field	Value
Publication number	US-2023004747-A1
Application number	US-202117365045-A
Country	US
Kind code	A1
Filing date	Jul 1, 2021
Priority date	Jul 1, 2021
Publication date	Jan 5, 2023
Grant date	—

Abstract

A system, method, and computer program product provides a way to separate connected or adhered adjacent characters of a digital image for license plate recognition. As a threshold processing, the method performs a recognition of character adhesion by obtaining character parameters using an image processor. The parameters include a horizontal max crossing and a ratio of width and height. A first rule-based module is used responsive to the character parameters to distinguish the adhered characters (character adhesions) that are easy to judge, leaving the uncertain part to a character adhesion classifier model for discrimination. Character adhesion data is obtained by data augmentation including the adding of a random distance between two single characters to create class like adhered characters. Then the character adhesion classifier model of single character and character adhesion data is trained. Any uncertain part can be distinguished by the trained character adhesion classifier model.

First claim

1 . A method implemented by at least one hardware processor comprising:
receiving, at the at least one hardware processor, a digital image comprising a sequence of characters; evaluating, implementing the at least one hardware processor, the digital image to determine a connectivity of one or more adjacent characters, said evaluating the digital image comprising:
processing, using the at least one hardware processor, the characters of the digital image to obtain character parameters; and
generating, using the at least one hardware processor, a connection indicator value of a character as a function of said character parameters, said generated character connection indicator indicating an uncertainty as to a character being connected to another character in the sequence; and responsive to the generated character connection indicator indicating an uncertainty as to a character being connected to another adjacent character in the sequence, said method further comprising:
running, using the at least one hardware processor, a character adhesion classifier model trained to recognize, from said image, a presence or not of connected adjacent character classes of adjacent adhered characters of the sequence; and
for connected adjacent characters, performing image processing using the at least one hardware processor to incrementally split and segment the connected characters; and performing, using the at least one hardware processor, character recognition to determine the sequence of characters including the segmented characters of the original digital image.

2 . The method of claim 1 , wherein said generated character connection indicator alternately indicates the character as being one of: a segmented character in the sequence, or a character connected to another adjacent character in the sequence.

3 . The method of claim 2 , wherein said character parameters of a character comprises:
a max horizontal crossing value of the character; a ratio value of the character, the ratio being a measure of a width/height of the character; and a means ratio being a function of the ratio of each of the characters of the sequence.

4 . The method of claim 2 , wherein responsive to the generated character connection indicator indicating the character as being a segmented character, said method further performing a character recognition of the character using a character recognition model.

5 . (canceled)

6 . The method of claim 1 , wherein responsive to the generated character connection indicator indicating a connection to another adjacent character in the sequence, said method further comprising:
running, using the at least one hardware processor, an image processor adapted to split the connected characters of the sequence.

7 . The method of claim 6 , wherein said running an image processor adapted to split the connected characters comprises:
performing, using the at least one hardware processor, a first character splitting method using adaptive thresholding for image binarization, said first character splitting method dynamically adjusting a segmenting threshold for distinguishing among two classes of data used for character segmentation; or performing a second character splitting method using a vertical histogram projection of said connected characters on an x-axis and locating a gap or trough on the projection as a location for character segmenting; or performing both the first character splitting method and second character splitting method for segmenting the connected characters.

8 . The method of claim 1 , wherein said training said character adhesion classifier model uses single alphabet and numeric characters labels, said method further comprising:
adding random distance between two single characters to create new connected adhesion character data, wherein said character adhesion classifier model is further trained using said single characters and said new connected adhesion character data.

9 . A system comprising at least one processor comprising hardware, the at least one processor configured to:
receive a digital image comprising a sequence of characters; evaluate the digital image to determine a connectivity of one or more adjacent characters wherein to evaluate the digital image, said at least one processor is further configured to:
process the characters of the digital image to obtain character parameters; and
generate a connection indicator value of a character as a function of said character parameters, said generated character connection indicator indicating an uncertainty as to a character being connected to another character in the sequence; and responsive to the generated character connection indicator indicating an uncertainty as to a character being connected to another adjacent character in the sequence, said at least one processor is further configured to:
run a character adhesion classifier model trained to recognize, from said image, a presence or not of connected adjacent character classes of adjacent adhered characters of the sequence; and
for connected adjacent characters, perform image processing to incrementally split and segment the connected characters; and perform character recognition to determine the sequence of characters including the segmented characters of the original digital image.

10 . The system of claim 9 , wherein
said generated character connection indicator alternatively indicating the character as being one of: a segmented character in the sequence, or a character connected to another character in the sequence.

11 . The system of claim 10 , wherein responsive to the generated character connection indicator indicating the character as being a segmented character, the at least one processor is further configured to:
perform a character recognition of the character using a character recognition model.

12 . The system of claim 10 , wherein
responsive to the generated character connection indicator indicating a connection to another adjacent character in the sequence, the at least one processor is further configured to: run an image processor adapted to split the connected characters of the sequence.

13 . The system of claim 12 , wherein to run an image processor adapted to split the connected characters, the at least one processor is further configured to:
perform a first character splitting method using adaptive thresholding for image binarization, said first character splitting method dynamically adjusting a segmenting threshold for distinguishing among two classes of data used for character segmentation; or perform a second character splitting method using a vertical histogram projection of said connected characters on an x-axis and locating a gap or trough on the projection as a location for character segmenting; or perform both the first character splitting method and second character splitting method for segmenting the connected characters.

14 . The system of claim 12 , wherein said training said character adhesion classifier model uses single alphabet and numeric characters labels, said at least one processor is further configured to:
add random distance between two single characters to create new connected adhesion character data, wherein said character adhesion classifier model is further trained using said single characters and said new connected adhesion character data.

15 . A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor comprising hardware, configures the at least one hardware processor to:
receive a digital image comprising a sequence of characters; evaluate

Assignees

• IBM

Inventors

• Zhu sheng nan
• Hu Guoqiang
• Ding Yuan Yuan
• Ji Peng
• Li Fan
• Xu Jian

Classifications

• G06F18/2431

  Multiple classes · CPC title

• G06N3/08

  Learning methods · CPC title

• G06V20/63 Primary

  Scene text, e.g. street names · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title

• G06V30/153

  using recognition of characters or words · CPC title