Systems and methods of using cached images to determine product counts on product storage structures of a product storage facility

Patent metadata

Field	Value
Publication number	US-2025329164-A1
Application number	US-202519254876-A
Country	US
Kind code	A1
Filing date	Jun 30, 2025
Priority date	Jan 24, 2023
Publication date	Oct 23, 2025
Grant date	—

Abstract

Systems and methods of detecting and recognizing products on product storage structures of a product storage facility include an image capture device that moves about and captures images of the product storage structures at the product storage facility. A computing device processes the obtained images to detect and identify the products on the product storage structure, crops each of the identified individual products from the image to generate a plurality of cropped images and generates an image histogram template, feature vector template and location information template for each of the cropped images. The cropped images are stored in an electronic database and represent a reference model for each of the identified individual products and are stored in association with the generated image histogram template, feature vector template and location information template to facilitate recognition of products subsequently captured on the product storage structure by the image capture device.

First claim

What is claimed is:

1 . A system comprising:
a processor; and a computer-readable medium storing instructions operative by the processor to:
obtain an image depicting a product;
separate the image into a plurality of image blocks;
calculate a histogram for each of the plurality of image blocks;
generate a histogram template for the image comprising the histogram for each of the plurality of image blocks;
generate a data schema for the image including the histogram template; and
store the data schema on an electronic database in association with the image, wherein the image is a reference model of a visual representation of the product and the data schema is stored on the electronic database to facilitate identification of the product from future images obtained by the processor.

2 . The system of claim 1 , wherein the computer-readable medium further stores instructions operative by the processor to:
analyze the image using a trained neural network to generate feature vectors for the image; generate a feature vector template for the image comprising the feature vectors; and generate the data schema to include the feature vector template.

3 . The system of claim 1 , wherein:
the image is captured by an image capture device in a product storage facility; and the computer-readable medium further stores instructions operative by the processor to:
generate a location information template including location information describing a location in the product storage facility where the image was captured; and
generate the data schema to include the location information template.

4 . The system of claim 1 , wherein the computer-readable medium further stores instructions operative by the processor to:
identify the product depicted in the image by detecting keywords in the image using optical character recognition; and save the image to the electronic database as the reference model for the product.

5 . The system of claim 1 , wherein the computer-readable medium further stores instructions operative by the processor to:
detect the product in the image using a computer vision model; crop the product from the image to form a cropped image; and use the cropped image in generating the plurality of image blocks.

6 . The system of claim 1 , wherein the future images are generated by an image capture device in capturing images of product storage areas in a product storage facility.

7 . The system of claim 1 , wherein the processor obtains the image by a transmission signal from an image capture device, the transmission signal received at a signal input operatively coupled to the processor.

8 . A method comprising:
obtaining, by a processor, an image depicting a product; separating, by the processor, the image into a plurality of image blocks; calculating, by the processor, a histogram for each of the plurality of image blocks; generating, by the processor, a histogram template for the image comprising the histogram for each of the plurality of image blocks; generating, by the processor, a data schema for the image including the histogram template; and storing, by the processor, the data schema on an electronic database in association with the image, wherein the image is a reference model of a visual representation of the product and the data schema is stored on the electronic database to facilitate identification of the product from future images obtained by the processor.

9 . The method of claim 8 , further comprising:
analyzing, by the processor, the image using a trained neural network to generate feature vectors for the image; generating, by the processor, a feature vector template for the image comprising the feature vectors; and generating, by the processor, the data schema to include the feature vector template.

10 . The method of claim 8 , wherein:
the image is captured by an image capture device in a product storage facility; and the method further comprises:
generating, by the processor, a location information template including location information describing a location in the product storage facility where the image was captured; and
generating, by the processor, the data schema to include the location information template.

11 . The method of claim 8 , further comprising:
identifying, by the processor, the product depicted in the image by detecting keywords in the image using optical character recognition; and saving, by the processor, the image to the electronic database as the reference model for the product.

12 . The method of claim 8 , further comprising:
detecting, by the processor, the product in the image using a computer vision model; cropping, by the processor, the product from the image to form a cropped image; and using, by the processor, the cropped image in generating the plurality of image blocks.

13 . The method of claim 8 , wherein the future images are generated by an image capture device in capturing images of product storage areas in a product storage facility.

14 . The method of claim 8 , wherein the processor obtains the image by a transmission signal from an image capture device, the transmission signal received at a signal input operatively coupled to the processor.

15 . A computer-readable medium storing instructions operative by a processor to:
obtain an image depicting a product; separate the image into a plurality of image blocks; calculate a histogram for each of the plurality of image blocks; generate a histogram template for the image comprising the histogram for each of the plurality of image blocks; generate a data schema for the image including the histogram template; and store the data schema on an electronic database in association with the image, wherein the image is a reference model of a visual representation of the product and the data schema is stored on the electronic database to facilitate identification of the product from future images obtained by the processor.

16 . The computer-readable medium of claim 15 , further storing instructions operative by the processor to:
analyze the image using a trained neural network to generate feature vectors for the image; generate a feature vector template for the image comprising the feature vectors; and generate the data schema to include the feature vector template.

17 . The computer-readable medium of claim 15 , wherein:
the image is captured by an image capture device in a product storage facility; and the computer-readable medium further stores instructions operative by the processor to:
generate a location information template including location information describing a location in the product storage facility where the image was captured; and
generate the data schema to include the location information template.

18 . The computer-readable medium of claim 15 , further storing instructions operative by the processor to:
identify the product depicted in the image by detecting keywords in the image using optical character recognition; and save the image to the electronic database as the reference model for the product.

19 . The computer-readable medium of claim 15 , further storing instructions operative by the processor to:
detect the product in the image using a computer vision model; crop the product from the image to form a cropped image; and use the cropped image in generating the plurality of image blocks.

20 . The computer-readable medium of claim 15 , wherein the future images are generated b

Assignees

• Walmart Apollo Llc

Inventors

• Duan Zhaoliang
• Ellison Benjamin R
• Zhang Lingfeng
• Jade Avinash M
• Balusu Raghava
• Yuan Mingquan
• Xiao Zhichun

Classifications

• G06F16/51

  Indexing; Data structures therefor; Storage structures · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06V20/63

  Scene text, e.g. street names · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title