Systems and methods of updating model templates associated with images of retail products at product storage facilities

Patent metadata

Field	Value
Publication number	US-2025307766-A1
Application number	US-202519233539-A
Country	US
Kind code	A1
Filing date	Jun 10, 2025
Priority date	Jan 30, 2023
Publication date	Oct 2, 2025
Grant date	—

Abstract

Systems and methods of updating templates for use in recognizing individual products in images captured at a product storage facility include an image capture device that captures one or more images of product storage structure at a product storage facility, a computing device in communication with the image capture device, and an electronic database that stores keyword model templates and feature model templates associated with images of previously recognized individual products detected at the product storage facility. The computing device obtains the keyword and feature model templates associated with a recognized product from the electronic database, extracts the keywords from the products associated with the obtained keyword model templates, identifies products that are similar to the recognized product, and updates the keyword model template for each of the products to include must keywords and negative keywords, facilitating recognition of products in subsequent images captured by the image capture device.

First claim

What is claimed is:

1 . A method comprising:
obtaining, by a processor, a plurality of images of a product storage area captured by an image capture device; extracting, by the processor using a convolutional neural network (CNN), a plurality of features from each of the plurality of images; generating, by the processor using the CNN, embeddings for each of the plurality of images based on the extracted plurality of features, wherein the embeddings include numerical representations of a corresponding image of the plurality of images based on the extracted plurality of features; generating, by the processor, a plurality of cluster nodes using the embeddings such that each of the plurality of cluster nodes is representative of one of the plurality of images; creating, by the processor, a cluster graph comprising the plurality of cluster nodes; selecting, by the processor, one of the plurality of cluster nodes as a centroid node and a predetermined number of surrounding nodes based on their proximity to the centroid node; and updating, by the processor, a feature model template to include images associated with the centroid node and the predetermined number of surrounding nodes, wherein the feature model template is utilized by the processor in identifying products from images.

2 . The method of claim 1 , further comprising:
performing, by the processor, optical character recognition (OCR) on the plurality of images to identify a product depicted in the plurality of images; and associating, by the processor, the feature model template with the identified product such that the feature model template is utilized by the processor in future processing of images for recognizing the identified product.

3 . The method of claim 2 , further comprising:
generating, by the processor, a universal product code (UPC) model for the identified product including the feature model template and keywords associated with the identified product, wherein the UPC model is configured to be utilized by the processor in future processing of images for recognizing the identified product.

4 . The method of claim 1 , further comprising:
identifying, by the processor, a plurality of clusters from the cluster graph each comprising a group of nodes from the plurality of cluster nodes, wherein each of the plurality of clusters corresponds to a distinct product.

5 . The method of claim 1 , wherein distances between each of the plurality of cluster nodes on the cluster graph are based on a level of similarity between the embeddings corresponding to the cluster node.

6 . The method of claim 5 , further comprising selecting, by the processor, the centroid node by:
calculating, for each of the plurality of cluster nodes, a sum of a distance between the cluster node and each of the other of the plurality of cluster nodes; and selecting, as the centroid node, a cluster node of the plurality of cluster nodes with a smallest sum of the distances.

7 . The method of claim 1 , wherein:
the image capture device is coupled to a motorized device operating in the product storage area; and the method further comprises sending controlling signals, by the processor, to the motorized device to move to a desired area within the product storage area for capturing images at the desired area.

8 . A system comprising:
a processor; and a computer-readable medium storing instructions that are operative by the processor to:
obtain a plurality of images of a product storage area captured by an image capture device;
extract, using a convolutional neural network (CNN), a plurality of features from each of the plurality of images;
generate, using the CNN, embeddings for each of the plurality of images based on the extracted plurality of features, wherein the embeddings include numerical representations of a corresponding image of the plurality of images based on the extracted plurality of features;
generate a plurality of cluster nodes using the embeddings such that each of the plurality of cluster nodes is representative of one of the plurality of images;
create a cluster graph comprising the plurality of cluster nodes;
select one of the plurality of cluster nodes as a centroid node and a predetermined number of surrounding nodes based on their proximity to the centroid node; and
update a feature model template to include images associated with the centroid node and the predetermined number of surrounding nodes, wherein the feature model template is utilized by the processor in identifying products from images.

9 . The system of claim 8 , wherein the computer-readable medium further stores instructions operative by the processor to:
perform optical character recognition (OCR) on the plurality of images to identify a product depicted in the plurality of images; and associate the feature model template with the identified product such that the feature model template is utilized by the processor in future processing of images for recognizing the identified product.

10 . The system of claim 9 , wherein the computer-readable medium further stores instructions operative by the processor to:
generate a universal product code (UPC) model for the identified product including the feature model template and keywords associated with the identified product, wherein the UPC model is configured to be utilized by the processor in future processing of images for recognizing the identified product.

11 . The system of claim 8 , wherein the computer-readable medium further stores instructions operative by the processor to:
identify a plurality of clusters from the cluster graph each comprising a group of nodes from the plurality of cluster nodes, wherein each of the plurality of clusters corresponds to a distinct product.

12 . The system of claim 8 , wherein distances between each of the plurality of cluster nodes on the cluster graph are based on a level of similarity between the embedding corresponding to the cluster node.

13 . The system of claim 12 , wherein the computer-readable medium further stores instructions operative by the processor to select the centroid node by:
calculating, for each of the plurality of cluster nodes, a sum of a distance between the cluster node and each of the other of the plurality of cluster nodes; and selecting, as the centroid node, a cluster node of the plurality of cluster nodes with a smallest sum of the distances.

14 . The system of claim 8 , wherein:
the image capture device is coupled to a motorized device operating in the product storage area; and the computer-readable medium further stores instructions operative by the processor to send controlling signals to the motorized device to move to a desired area within the product storage area for capturing images at the desired area.

15 . A computer-readable medium storing instructions operative by a processor to:
obtain a plurality of images of a product storage area captured by an image capture device; extract, using a convolutional neural network (CNN), a plurality of features from each of the plurality of images; generate, using the CNN, embeddings for each of the plurality of images based on the extracted plurality of features, wherein the embeddings include numerical representations of a corresponding image of the plurality of images based on the extracted plurality of features; generate a plurality of cluster nodes using the embeddings such that each of the plurality of cluster nodes is representative of one of the plurality of images; create a cluster graph comprising the plurality of cluster nodes; select one of the plurali

Assignees

• Walmart Apollo Llc

Inventors

• Zhang Han
• Pachauri Abhinav
• Balusu Raghava
• Ghosh Ashlin
• Jade Avinash M
• Zhang Lingfeng
• Muktevi Srinivas
• Jhunjhunwala Amit
• Duan Zhaoliang

Classifications

• G06V20/60 Primary

  Type of objects · CPC title

• G06V2201/07

  Target detection · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title