Systems and methods for auto-replenishment of products using pressure mapping sensors and machine learning

Patent metadata

Field	Value
Publication number	US-12400182-B2
Application number	US-202318381497-A
Country	US
Kind code	B2
Filing date	Oct 18, 2023
Priority date	Aug 27, 2020
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

Systems and methods for identifying, tracking usage of, and replenishing products on a shelf of a product storage unit include a pressure sensor array positioned on the shelf and configured to detect forces exerted on the pressure sensor array by each of the products located on the shelf. An electronic database stores reference pressure array data representative of a reference pressure data model relative to each of the products. A computing device obtains a pressure data set associated with a product that was captured by the pressure sensor array when the product was positioned on the pressure sensor array, and correlates the pressure data set associated with the product with the reference pressure array data stored in the electronic database to determine an identity of the product and a consumption level of the product, automatically replenishing the product when consumption of the product is above a set threshold.

First claim

What is claimed is:

1. A system for identifying, tracking usage of, and replenishing products on a shelf of a product storage unit, the system comprising:
a pressure sensor array positioned on the shelf and configured to detect forces exerted on the pressure sensor array by each of the products located on the shelf;
an electronic database configured to store reference pressure array data representative of a reference pressure data model relative to each of the products;
a computing device in communication with the pressure sensor array and the electronic database;
the reference pressure data model for a selected one of the products being generated via training, by the computing device, of a machine learning model using pressure data generated on the pressure sensor array by a reference product substantially identical to the selected one of the products when the reference product is at a plurality of consumption levels;
each reference pressure data model generated via training of the machine learning model being associated in the electronic database with product identity data that identifies a product corresponding to the reference pressure data model; and
the computing device including a processor-based control circuit configured to:
activate the pressure sensor array to cause the pressure sensor array to generate a pressure reading for the selected one of the products when the selected one of the products is positioned on the pressure array;
obtain a pressure data set associated with the selected one of the products and captured by the pressure sensor array when the selected one of the products was positioned on the pressure sensor array;
process the pressure data set associated with the selected one of the products and captured by the pressure sensor array when the selected one of the products was positioned on the pressure sensor array by generating a boxed pressure data set, where a virtual bounding box extends around the pressure data set associated with the selected one of the products;
correlate the boxed pressure data set associated with the selected one of the products with the reference pressure array data stored in the electronic database;
in response to a determination by the control circuit that the reference pressure array data stored in the electronic database includes a reference pressure data model generated via the training of the machine learning model that matches the boxed pressure data set associated with the selected one of the

products, obtain the product identity data stored in the electronic database in association with the reference pressure data model that matches the boxed pressure set associated with the selected one of the products to determine an identity of the selected one of the products; and
in response to a determination by the control circuit that the reference pressure array data stored in the electronic database does not include the reference pressure data model generated via the training of the machine learning model does not match the boxed pressure data set associated with the selected one of the products:
cause the selected one of the products to be scanned by a sensor configured to scan identifying indicia on the selected one of the products;
obtain the identifying indicia scanned by the sensor and, based on a determination of the identity of the selected one of the products, generate product identity data identifying the selected one the products;
update the trained machine learning model that is stored in the electronic database associated with the selected one of the products such that an updated machine learning model includes the an updated boxed pressure set associated with the selected one of the products the identity of which was determined based on the scan of the identifying indicia of the selected one of the products; and
identify, by the updated machine learning model, a subsequent product identical to the selected one of the products based on a match between the pressure data generated on the pressure sensor array by the subsequent product when the subsequent product is located on the shelf and the updated boxed pressure set included in the updated machine learning model.

2. The system of claim 1 , wherein the product-storing unit comprises a refrigerator, a freezer, a cabinet, a cupboard, a pantry, and a shelving structure.

3. The system of claim 1 , wherein the electronic database further comprises electronic data representative of: pressure data sets previously obtained by the pressure sensor array, identity of the products on or likely to be on the shelf, relative orientation of the products on the shelf, price of the products, unit weight of each of the products, unit volume of each of the products, sensor configuration, relative sensor location, and unit quantity of each of the products.

4. The system of claim 1 , wherein, based one or more pressure data sets obtained by the computing device from the pressure sensor array, the control circuit of the computing device is configured to determine that:
the selected one of the products is no longer positioned on the pressure sensor array; or
the selected one of the products was moved to a different location on the pressure sensor array.

5. The system of claim 1 , wherein, based on a determination of the identity of the selected one of the products by the control circuit of the computing device, the control circuit is programmed to correlate the pressure data set associated with the selected one of the products with the reference pressure array data stored in the electronic database that matches the selected one of the products to determine a level of consumption of the selected one of the products.

6. The system of claim 5 ,
wherein the electronic database further comprises product replenishment threshold settings associated with a user; and
wherein the control circuit of the computing device is configured, in response to a determination, by the computing device, that the level of consumption of the selected one of the products is above the product replenishment threshold, to:
automatically process a replenishment order for the selected one of the products for delivery to the user; and
cause transmission of an electronic notification to an electronic device of the user, the electronic notification including a confirmation that the replenishment order was processed.

7. The system of claim 6 ,
wherein the electronic database further comprises a minimum total dollar amount threshold for triggering a delivery of the replenishment order; and
wherein the control circuit of the computing device is configured to:
automatically process the replenishment order for the selected one of the products in response to a determination, by the computing device, that the minimum total dollar amount threshold is exceeded; and
in response to a determination, by the computing device, that the minimum total dollar amount threshold is not exceeded, delay processing the replenishment order for the selected one of the products until products are added to the replenishment order such that the minimum total dollar amount threshold of the replenishment order is exceeded.

8. The system of claim 5 ,
wherein the electronic database further comprises product replenishment threshold settings associated with a user; and
wherein the control circuit of the computing device is configured, in response to a determination, by the computing device, that the level of consumption of the selected one of the products is above the product replenishment threshold, to:
generate a proposed replenishment order for the selected one of the products for delivery to the user; and
cause transmission of an electronic notification to an electronic device of the user, the e

Assignees

• Walmart Apollo Llc

Inventors

• Barton Elizabeth J
• Larson Soren A
• Salehian-Dardashti Soheil
• Moeller-Chan Victoria A

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06Q10/06315 Primary

  Needs-based resource requirements planning or analysis · CPC title

• F25D9/00

  Devices not associated with refrigerating machinery and not covered by groups F25D1/00 - F25D7/00; Combinations of devices covered by two or more of the groups F25D1/00 - F25D7/00 · CPC title