Systems and methods for checking and confirming the purchase of merchandise items

Patent metadata

Field	Value
Publication number	US-12373873-B2
Application number	US-202418641522-A
Country	US
Kind code	B2
Filing date	Apr 22, 2024
Priority date	Nov 12, 2019
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

There are provided systems and methods relating to checking and confirming merchandise purchased at shopping facilities. In one form, the system includes: a shopping cart containing merchandise items to be purchased; a sales transaction database; a point-of-sales system that creates transaction records identified by transaction identifiers; and a mobile device used by an employee that includes a sensor to scan a paper or digital receipt to obtain the transaction identifier and a camera to capture images of the items in the shopping cart. The system also includes a control circuit that receives the transaction identifier, accesses the database using the identifier to determine the purchased items; analyzes the images of the merchandise items in the shopping cart and creates a computer vision receipt listing detected items; compares the purchased items with the detected items; and takes an action if there is a discrepancy.

First claim

What is claimed is:

1. A system comprising:
a point-of-sales system of a retailer that configured to create an electronic transaction record of a sales transaction of purchased merchandise items that is identified by a barcode or QR code;
a mobile electronics device of a retailer configured to:
determine the purchased merchandise items using the barcode or QR code;
capture an image sequence of merchandise items in a shopping cart;
analyze the image sequence of the merchandise items in the shopping cart using a computer vision algorithm to identify the merchandise items based on the image sequence and create a computer vision receipt listing the merchandise items identified from the image sequence;
identify a discrepancy by comparing the purchased merchandise items indicated by the barcode or QR code with the identified merchandise items from the computer vision receipt;
display a list of unpaid merchandise items or images of unpaid merchandise items associated with the discrepancy; and
transmit a control signal to a retrieval device of the retailer with instructions for retrieving an unpaid merchandise item associated with the discrepancy, wherein the retrieval device is configured to retrieve the unpaid merchandise item in response to receiving the control signal.

2. The system of claim 1 , wherein the mobile electronics device of the retailer analyzes the image sequence using one or more convolutional neural networks (CNNs).

3. The system of claim 2 , wherein the mobile electronics device of the retailer compares the image sequence with images of known products to identify the merchandise items.

4. The system of claim 2 , wherein the mobile electronics device of the retailer is configured identifies the merchandise items using identification information on merchandise items to identify the merchandise items.

5. The system of claim 4 , wherein the identification information includes numbers or shapes identifying certain products.

6. The system of claim 1 , wherein the retrieval device comprises an automated vehicle of the retailer configured to receive the control signal for retrieving the unpaid merchandise item.

7. The system of claim 6 , wherein the automated vehicle is an automated aerial drone.

8. The system of claim 6 , wherein the automated vehicle is an automated ground vehicle.

9. A method comprising:
creating, by a point-of-sales system of a retailer, an electronic transaction record of a sales transaction of purchased merchandise items that is identified by a barcode or QR code;
determining, by a mobile electronics device of the retailer, the purchased merchandise items using the barcode or QR code;
capturing, by the mobile electronics device of the retailer, an image sequence of merchandise items in a shopping cart;
analyzing, by the mobile electronics device of the retailer, the image sequence of the merchandise items in the shopping cart using a computer vision algorithm to identify the merchandise items based on the image sequence and creating a computer vision receipt listing the merchandise items identified from the image sequence;
identifying, by the mobile electronics device of the retailer, a discrepancy by comparing the purchased merchandise items indicated by the barcode or QR code with the identified merchandise items from the computer vision receipt;
displaying, by the mobile electronics device of the retailer, a list of unpaid merchandise items or images of unpaid merchandise items associated with the discrepancy; and
transmitting, by the mobile electronics device of the retailer, a control signal to a retrieval device of the retailer with instructions for retrieving an unpaid merchandise item associated with the discrepancy, wherein the retrieval device is configured to retrieve the unpaid merchandise item in response to receiving the control signal.

10. The method of claim 9 , wherein the mobile electronics device of the retailer analyzes the image sequence using one or more convolutional neural networks (CNNs).

11. The method of claim 10 , wherein the mobile electronics device of the retailer compares the image sequence with images of known products to identify the merchandise items.

12. The method of claim 10 , wherein the mobile electronics device of the retailer identifies the merchandise items using identification information on merchandise items to identify the merchandise items.

13. The method of claim 12 , wherein the identification information includes numbers or shapes identifying certain products.

14. The method of claim 9 , wherein the retrieval device comprises an automated vehicle of the retailer, the control signal instructing the automated vehicle to retrieve the unpaid merchandise item.

15. The method of claim 14 , wherein the automated vehicle is an automated aerial drone.

16. The method of claim 14 , wherein the automated vehicle is an automated ground vehicle.

17. A mobile electronics device of a retailer comprising:
at least one processor; and
at least one memory comprising computer-readable instructions, the at least one processor, the at least one memory and the computer-readable instructions configured to cause the at least one processor to:
receive, from a point-of-sales system of a retailer, an electronic transaction record of a sales transaction of purchased merchandise items that is identified by a barcode or QR code;
determine the purchased merchandise items using the barcode or QR code;
capture an image sequence of merchandise items in a shopping cart;
analyze the image sequence of the merchandise items in the shopping cart using a computer vision algorithm to identify the merchandise items based on the image sequence and creating a computer vision receipt listing the merchandise items identified from the image sequence;
identify a discrepancy by comparing the purchased merchandise items indicated by the barcode or QR code with the identified merchandise items from the computer vision receipt;
display a list of unpaid merchandise items or images of unpaid merchandise items associated with the discrepancy; and
transmit a control signal to a retrieval device of the retailer with instructions for retrieving an unpaid merchandise item associated with the discrepancy, wherein the retrieval device is configured to retrieve the unpaid merchandise item in response to receiving the control signal.

18. The mobile electronics device of claim 17 , wherein the image sequence is analyzed by comparing the image sequence with images of known products to identify the merchandise items.

Assignees

• Walmart Apollo Llc

Inventors

• Xiao Zhichun
• Zhang Lingfeng
• Liu Yao
• Hammer Jon
• Tang Yutao
• Fang Sicong
• Liu Haining
• Sun Yijing
• Yuan Mingquan
• Zhang Shouyi
• Yu Pingjian
• Reagan Ryan B
• Mao Tianyi
• Susseelaa Shangeetha Ravichandran
• Wang Zhenyu
• Zhu Feiyun

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06Q30/0633

  Managing shopping lists, e.g. compiling or processing purchase lists (shipping orders G06Q10/083; order filling G06Q10/087) · CPC title

• G06Q20/209

  Specified transaction journal output feature, e.g. printed receipt or voice output · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06Q20/047

  using payment protocols involving electronic receipts · CPC title