Systems and methods of monitoring location labels of product storage structures of a product storage facility

Patent metadata

Field	Value
Publication number	US-12437263-B2
Application number	US-202318203499-A
Country	US
Kind code	B2
Filing date	May 30, 2023
Priority date	May 30, 2023
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

Systems and methods of monitoring location labels on product storage structures of a product storage facility include an image capture device that captures images of the product storage structures and a computing device programmed to analyze the images of the product storage structures captured by the image capture device to detect location labels located on the product storage structures. Based on detection that one or more location labels located on the product storage structures are associated with an error condition, the computing device generates a location label alert indicating at least one location label that requires a location label check by a worker at the product storage facility. A mobile application executable on a device of the worker at the product storage facility displays a user interface that lists location labels alerts and permits the worker to print replacement labels for product structures associated with the alerts.

First claim

What is claimed is:

1. A system for monitoring location labels on product storage structures of a product storage facility, the system comprising:
an image capture device having a field of view that includes a product storage structure at the product storage facility configured to have products arranged thereon, wherein the image capture device is configured to capture at least one image of the product storage structure, and wherein the image capture device comprises a motorized robotic unit that autonomously moves around the product storage facility;
a computing device including a control circuit, the computing device communicatively coupled to the image capture device, the control circuit being configured to:
determine movement instructions for the motorized robotic unit;
autonomously control the motorized robotic unit by sending the movement instructions, wherein the motorized robotic unit enables the image capture device to capture the at least one image of the product storage structure in sequence;
analyze the at least one image of the product storage structure captured by the image capture device to detect at least one location label located on the product storage structure, wherein a trained machine learning model is used to detect the at least one location label; and
based on detection that the at least one location label located on the product storage structure is at least one of missing, incomplete, damaged, deformed, and at least partially obstructed, generate a location label alert indicating that the at least one location label requires a location label check by a worker at the product storage facility; and

a mobile application executable on a user device of the worker at the product storage facility, the mobile application, when executed, is configured to cause a user interface to be displayed to the worker on a display of the user device, wherein the user interface lists at least one location label alert generated by the control circuit of the computing device, wherein the user interface permits the worker to:
view a location of the product storage structure associated with the location label alert;
input an inspection result of a physical inspection of the at least one location label of the product storage structure, wherein the inspection result of the physical inspection is used to retrain the trained machine learning model;
generate a replacement label for the product storage structure associated with the location label alert and output signaling to cause the replacement label to be printed; and
scan the replacement label to verify that the worker affixed the replacement label to the product storage structure associated with the location label alert.

2. The system of claim 1 ,
wherein the user interface is configured such that each location label alert listed within the user interface is selectable by the worker; and
wherein, in response to a location label alert being selected by the worker within the user interface, the user interface is configured to display at least a portion of the at least one image of the product storage structure associated with the location label alert selected by the worker.

3. The system of claim 2 , wherein, in response to the location label alert being selected by the worker within the user interface, the user interface is configured to display a listing of the products stored on the product storage structure associated with the location label alert selected by the worker.

4. The system of claim 1 , wherein the user interface includes selectable fields configured to permit the worker to manually input the location of the product storage structure associated with the location label alert.

5. The system of claim 1 , wherein the user interface is configured to permit the worker to select a portable printer for printing the replacement label for the product storage structure associated with the location label alert.

6. The system of claim 1 , wherein the user interface is configured to:
generate a notification to the worker that the replacement label has been printed; and
generate a location label scanning feature on the display of the user device to permit the worker to scan the replacement label to verify that the worker affixed the replacement label to the product storage structure associated with the location label alert.

7. The system of claim 1 , wherein the image capture device comprises a camera to permit the motorized robotic unit to capture the at least one image of the product storage structure.

8. The system of claim 1 , wherein the control circuit is further configured to process the at least one image to:
generate virtual boundary lines that surround the at least one location label detected in the at least one image; and
extract one or more characters from the at least one location label detected in the at least one image.

9. The system of claim 1 , further comprising an electronic database configured to store the at least one image captured by the image capture device, and wherein the control circuit is programmed to transmit the location label alert to the electronic database for storage.

10. The system of claim 9 , wherein the user interface is further configured to cause the user device of the worker to transmit a notification to at least one of the computing device and the electronic database, the notification indicating that the worker scanned the replacement label to verify that the worker affixed the replacement label to the product storage structure associated with the location label alert.

11. A method of monitoring location labels on product storage structures of a product storage facility, the method comprising:
capturing at least one image of a product storage structure at the product storage facility configured to have products arranged thereon by an image capture device having a field of view that includes the product storage structure at the product storage facility, wherein the image capture device comprises a motorized robotic unit that autonomously moves around the product storage facility;
by a computing device including a control circuit and communicatively coupled to the image capture device:
determining movement instructions for the motorized robotic unit;
autonomously controlling the motorized robotic unit by sending the movement instructions, wherein the motorized robotic unit enables the image capture device to capture the at least one image of the product storage structure in sequence;
analyzing the at least one image of the product storage structure captured by the image capture device to detect at least one location label located on the product storage structure, wherein a trained machine learning model is used to detect the at least one location label; and
based on detection that the at least one location label located on the product storage structure is at least one of missing, incomplete, damaged, deformed, and at least partially obstructed, generating a location label alert indicating that the at least one location label requires a location label check by a worker at the product storage facility; and

by a mobile application executable on a user device of the worker at the product storage facility, causing a user interface to be displayed to the worker on a display of the user device, wherein the user interface lists at least one location label alert generated by the control circuit of the computing device, wherein the user interface permits the worker to:
view a location of the product storage structure associated with the location label alert;
input an inspection result of a physical inspection of the at least one location label of the product storage structure, wherein the inspecti

Assignees

• Walmart Apollo Llc

Inventors

• Zhao Chongrui
• Narayanan Shanthi
• Benson Tracy E

Classifications

• G06K15/024

  for printing on segmented surfaces, e.g. sticker sheets, label rolls · CPC title

• G06K7/1413

  1D bar codes · CPC title

• G06V30/10

  Character recognition · CPC title

• G06V20/63

  Scene text, e.g. street names · CPC title

• G06V20/50

  Context or environment of the image · CPC title