Object tracking and cluster voting for tag detections and exceptions

Patent metadata

Field	Value
Publication number	US-2025218000-A1
Application number	US-202418403452-A
Country	US
Kind code	A1
Filing date	Jan 3, 2024
Priority date	Jan 3, 2024
Publication date	Jul 3, 2025
Grant date	—

Abstract

Examples enable pallet tag tracking and cluster voting for more accurate pallet tag management using images of a selected pallet. A tag manager tracks a pallet through multiple images of the pallet to ensure the same pallet appears in every image. If the pallet tag is absent from all the images, a tag missing confidence score is generated that indicates the degree of confidence that the tag is missing from the pallet and not merely out of view. The score is used to prioritize handling of pallet tag missing exceptions. If the pallet tag is present in the images, optical character recognition (OCR) results for each tag image are aggregated into a tag cluster with a confidence score calculated for each result. A pallet tag identification (ID) number is predicted based on the result having the highest confidence score to ensure the pallet tag ID is complete and accurate.

First claim

What is claimed is:

1 . A system for pallet tag tracking and cluster voting, the system comprising:
a processor; and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
obtain pallet and pallet tag detection results associated with a plurality of pallets within a plurality of images generated by an image capture device within a retail environment;
track a selected pallet appearing within a set of images within the plurality of images, the set of images comprising a sequence of images including a portion of the selected pallet;
analyze the set of images to determine whether a pallet tag associated with the selected pallet is present within any image in the set of images;
assign a confidence score indicating a degree of confidence the pallet tag associated with the selected pallet is absent, wherein a pallet tag missing exception is generated in response to the determination the pallet tag is absent, wherein handling the pallet tag missing exception is prioritized based on the confidence score; and
calculate a cluster voting score for each pallet tag text recognition result in a set of text recognition results associated with a set of pallet tag detections for the selected pallet in response to the determination the pallet tag is present within the set of images, wherein a pallet tag text recognition result having a highest score is used to identify an accurate tag identification (ID) number on the pallet tag.

2 . The system of claim 1 , wherein the instructions are further operative to:
responsive to identifying the pallet tag in an image in the set of images, generate the pallet tag text recognition result based on optical character recognition (OCR) on the image, wherein the pallet tag text recognition result comprises at least a portion of a pallet ID number.

3 . The system of claim 1 , wherein the instructions are further operative to:
responsive to identifying the pallet tag in a set of two images from the set of images that includes a portion of the pallet tag, generate a first pallet tag text recognition result based on a first image from the set of two images that includes the pallet tag and a second pallet tag text recognition result based on a second image from the set of two images; generate a first confidence score associated with the first pallet tag text recognition result and a second confidence score associated with the second pallet tag text recognition result; and identify a pallet ID of the selected pallet based on the first pallet tag text recognition result associated with a highest confidence score, wherein the second pallet tag text recognition result associated with a lowest confidence score is disregarded.

4 . The system of claim 1 , wherein the instructions are further operative to:
responsive to identifying the pallet tag in a cluster of images from the set of images, generate a cluster of pallet tag text recognition results corresponding to the cluster of images; calculate a combined confidence score for each repeated instance of a pallet ID identified in the cluster of pallet tag text recognition results, the combined confidence score comprising an identical pallet ID count times a confidence score; and select a pallet ID associated with a highest combined confidence score.

5 . The system of claim 1 , wherein the instructions are further operative to:
obtain a plurality of pallet tag missing exceptions associated with the plurality of pallets; generate a plurality of confidence scores associated with the plurality of pallet tag missing exceptions; and rank each pallet tag missing exception based on the plurality of confidence scores, wherein the plurality of pallet tag missing exceptions is resolved in accordance with the rank for each pallet tag missing exception.

6 . The system of claim 1 , wherein the instructions are further operative to:
detect the plurality of pallets and pallet tags by a trained object detection model based on the plurality of images; enclose the selected pallet within a large bounding box in each image in the set of images by the trained object detection model; and enclose each pallet tag associated with the selected pallet within a small bounding box in image data associated with the set of images, wherein the image data is used for tracking the selected pallet through the sequence of images.

7 . The system of claim 1 , wherein the instructions are further operative to:
calculate a confidence score for each pallet tag text recognition result in a plurality of pallet tag text recognition results for the pallet tag; apply a threshold minimum confidence score to the plurality of pallet tag text recognition results; and filter any pallet tag text recognition results having a score that is less than the threshold minimum confidence score.

8 . A method for pallet tracking and cluster voting, the method comprising:
obtaining pallet and pallet tag detection results associated with a plurality of pallets within image data associated with a plurality of images generated by an image capture device within a retail facility; tracking a selected pallet appearing within a set of images within the plurality of images, the set of images comprising images including a portion of the selected pallet; determining whether a pallet tag associated with the selected pallet is present within any image in the set of images using a set of coordinates associated with the pallet; responsive to determining the pallet tag is absent from the set of images, generating a tag missing confidence score indicating a degree of confidence the pallet tag associated with the selected pallet is absent based on quality of image data associated with the set of images; and triggering a pallet tag missing exception associated with the selected pallet, the pallet tag missing exception including the tag missing confidence score, wherein the pallet tag missing exception is prioritized based on the tag missing confidence score.

9 . The method of claim 8 , further comprising:
calculating a cluster voting score for each pallet tag text recognition result in a set of text recognition results associated with a set of pallet tag detections for the selected pallet in response to presence of the pallet tag within the set of images, wherein a pallet tag text recognition result having a highest score is used to identify an accurate tag identification (ID) number on the pallet tag.

10 . The method of claim 8 , further comprising:
responsive to identifying the pallet tag in a set of two images from the set of images that includes a portion of the pallet tag, generating a first pallet tag text recognition result based on a first image from the set of two images that includes the pallet tag and a second pallet tag text recognition result based on a second image from the set of two images; generating a first confidence score associated with the first pallet tag text recognition result and a second confidence score associated with the second pallet tag text recognition result; and identifying a pallet ID of the selected pallet based on the first pallet tag text recognition result associated with a highest confidence score, wherein the second pallet tag text recognition result associated with a lowest confidence score is disregarded.

11 . The method of claim 8 , further comprising:
assigning a first rank to a first pallet tag missing exception based on a first tag missing confidence score; assigning a second rank to a second pallet tag missing exception based on a second tag missing, wherein the first rank is a higher priority rank than the second rank; and assigning a user to resolve the firs

Assignees

• Walmart Apollo Llc

Inventors

• Zhang Han
• Jade Avinash Madhusudanrao
• Zhang Lingfeng
• Duan Zhaoliang
• Yuan Mingquan
• Rader Eric W
• Huang Zhiwei
• Ellison Benjamin
• Robinson William Craig
• Chakraborty Siddhartha
• Balusu Raghava
• Gupta Aadarsh
• Wang Jing
• Chai Rongdong
• Ghosh Ashlin
• Viatchaninov Oleksandr

Classifications

• G06V30/10

  Character recognition · CPC title

• G06V30/30

  based on the type of data · CPC title

• G06V30/19107

  Clustering techniques · CPC title

• G06V20/50

  Context or environment of the image · CPC title

• G06V20/62 Primary

  Text, e.g. of license plates, overlay texts or captions on TV images · CPC title