Pallet tag classification for improved text recognition accuracy

Patent metadata

Field	Value
Publication number	US-12586031-B2
Application number	US-202318537949-A
Country	US
Kind code	B2
Filing date	Dec 13, 2023
Priority date	Dec 13, 2023
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Examples provide for pallet classification and pallet tag text recognition. The system includes a pallet text manager that classifies a type of pallet tag based on detected lines of text in the pallet tag using a classification model. Qualified lines of text are selected from the detected lines of text based on the classification type and corresponding format of the text. Each qualified line of text is associated with a pallet attribute, such as a pallet identifier (ID), an item ID, or a date of creation of the pallet tag. Attribute values from the set of qualified lines of text are paired with location data for the current location of the pallet. The attribute values and the paired location data are saved in a pallet attribute table. The pallet attributes are used to identify the location of pallets in a retail facility with improved accuracy and efficiency.

First claim

What is claimed is:

1 . A system for pallet text recognition with improved accuracy, the system comprising:
a data storage device; a processor; and a computer-readable medium storing instructions that are operative upon execution by the processor to: execute a pallet text manager comprising at least a neural network (NN) recognition model, an NN classification model, and an NN location recognition model, wherein the NN recognition model is trained for at least small text recognition and the NN classification model is trained for at least pallet attribute recognition, the pallet text manager being operable to: detect, using the NN recognition model, a plurality of lines of text associated with an image of a pallet tag associated with a pallet, wherein the plurality of lines of text are associated with a first tag format, and wherein the image comprises a plurality of pixels; determine, using the NN recognition model, whether one or more lines of text within the plurality of lines of text are tilted within a portion of the image, and based on the determination that one or more the plurality of lines of text are tilted:
generate a plurality of text-related values associated with a plurality of text-related pixels in the portion of the image and a plurality of non-text related values associated with a plurality of non-text related pixels in the portion of the image, the plurality of text-related values corresponding to alphanumeric characters in a tilted line of text; and
enclose the plurality of text-related values within a quadrilateral shaped bounding box, wherein the quadrilateral shaped bounding box encloses the plurality of text-related pixels detected on the pallet tag;
classify, using the NN classification model, a type of the pallet tag based on the plurality of lines of text, the type of the pallet tag corresponding to the first tag format; select, using the NN classification model, a set of qualified lines of text within the plurality of lines of text using the first tag format associated with the type of the pallet tag by applying a stored first set of rules associated with the first tag format to identify a qualified lines of text, wherein the qualified lines of text comprising a pallet attribute; extract a set of pallet attribute values from the set of qualified lines of text, the set of pallet attribute values comprising a pallet identifier (ID); and map the set of pallet attribute values to a location ID in a pallet entry within a pallet attributes table stored in the data storage device, wherein the location ID identifies a location of the pallet within a retail facility, wherein the location ID is being generated by the NN location recognition model, and wherein the location of the pallet is presented to a user via a user interface device.

2 . The system of claim 1 ,
wherein the quadrilateral shaped bounding box is color-coded to indicate items recognized in the image, and wherein the pallet text manager interprets the color-coded indicators according to a stored set of rules associated with the first tag format.

3 . The system of claim 1 , wherein the instructions are further operative to:
identify a set of disqualified lines of text within the plurality of lines of text; and filter the set of disqualified lines of text from the plurality of lines of text to reduce noise within the plurality of lines of text.

4 . The system of claim 1 , wherein the set of pallet attribute values comprises an item ID associated with an item, wherein the instructions are further operative to:
pair the item ID with the location of the pallet; and store the item ID paired with the location ID in the pallet entry within the pallet attributes table.

5 . The system of claim 1 , wherein the set of pallet attribute values comprises a date of creation of the pallet tag, wherein the instructions are further operative to:
pair the date of creation of the pallet tag with the location of the pallet; and store the date of creation paired with the location ID in the pallet entry within the pallet attributes table.

6 . The system of claim 1 , wherein the instructions are further operative to:
fine-tune a set of hyperparameters associated with a recognition model implemented on the processor, wherein the recognition model is trained on a first version of a data set, and wherein the recognition model is capable of analyzing pallet tags associated with a second version of the data set.

7 . The system of claim 1 , wherein the instructions are further operative to:
detect a second plurality of lines of text associated with a second pallet tag having a second tag format; classify the type of the second pallet tag corresponding to the second tag format; identify a second set of qualified lines of text within the second plurality of lines of text using the second tag format; and assign the second set of qualified lines of text to a second set of pallet attributes in the pallet attributes table, the set of pallet attribute values comprising a second pallet ID, an item ID, and a date of creation of the second pallet tag.

8 . A method for pallet text recognition with improved accuracy, the method comprising:
receiving a portion of an image associated with a pallet tag associated with a pallet within a retail facility, the image generated by an image capture device within the retail facility; executing a pallet text manager comprising at least a neural network (NN) recognition model, an NN classification model, and an NN location recognition model, wherein the NN recognition model is trained for at least small text recognition and the NN classification model is trained for at least pallet attribute recognition, the pallet text manager being operable to: detecting, using the NN recognition model, a plurality of lines of text associated with the portion of the image of the pallet tag associated with the pallet, wherein the plurality of lines of text are associated with a first tag format, and wherein the image comprises a plurality of pixels; determining, using the NN recognition model, whether one or more lines of text within the plurality of lines of text are tilted within a portion of the image, and based on the determination that one or more the plurality of lines of text are tilted:
generating a plurality of text-related values associated with a plurality of text-related pixels in the portion of the image and a plurality of non-text related values associated with a plurality of non-text related pixels in the portion of the image, the plurality of text-related values corresponding to alphanumeric characters in a tilted line of text; and
enclosing the plurality of text-related values within a quadrilateral shaped bounding box, wherein the quadrilateral shaped bounding box encloses the plurality of text-related pixels detected on the pallet tag;
classifying, using the NN classification model, a type of the pallet tag based on the plurality of lines of text, the type of the pallet tag corresponding to the first tag format; selecting, using the NN classification model, a set of qualified lines of text within the plurality of lines of text using the first tag format associated with the type of the pallet tag by applying a stored first set of rules associated with the first tag format to identify a qualified lines of text, wherein the qualified lines of text comprising a pallet attribute; extracting a set of pallet attribute values from the set of qualified lines of text, the set of pallet attribute values comprising a pallet identifier (ID); and mapping the set of pallet attribute values to a location ID in a pallet entry within a pallet attributes table stored in a data storage device, wherein the location ID identifies a location of

Assignees

• Walmart Apollo Llc

Inventors

• Chen Yilun
• Duan Zhaoliang
• Zhang Lingfeng
• Yuan Mingquan
• Robinson William Craig
• Arora Ishan
• Ellison Benjamin
• Rader Eric W
• Siler Elizabeth Ann
• Zhang Han
• Pachauri Abhinav
• Chakraborty Siddhartha
• Balusu Raghava
• Ghosh Ashlin
• Jade Avinash Madhusudanrao
• Anand Subhash
• Gupta Aadarsh
• Lobo Paul
• Shah Ketan
• Huang Zhiwei
• Wang Jing
• Chai Rongdong

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title