Methods, systems, articles of manufacture, and apparatus for decoding purchase data using an image

Patent metadata

Field	Value
Publication number	US-12229741-B2
Application number	US-202217710660-A
Country	US
Kind code	B2
Filing date	Mar 31, 2022
Priority date	Jun 24, 2021
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed that decode purchase data using an image. An example apparatus includes a dictionary including associated product descriptions and barcodes, interface circuitry, and processing circuitry to execute machine readable instructions to obtain purchase details and barcodes corresponding to a receipt, the purchase details including receipt product descriptions, generate a search query that includes a first receipt product description of the receipt product descriptions, a list of barcodes corresponding to the barcodes, and a store identifier associated with the receipt, execute a search against the dictionary using the search query to identify a barcode from the list of barcodes that corresponds to the first receipt product description, and in response to identifying the barcode that corresponds to the first receipt product description, associating the barcode and the first receipt product description and adding the association to the dictionary.

First claim

What is claimed is:

1. An apparatus comprising:
a first dictionary including images of product descriptions and barcodes;
interface circuitry;
machine readable instructions; and
at least one processor circuit to be programmed by the machine readable instructions to:
train a region detection machine learning model based on (a) the images of the product descriptions and barcodes, (b) ground truth annotations, and (c) intersection over union (IoU) comparisons of bounding boxes associated with the images;
based on satisfying a threshold IoU value of the region detection machine learning model, obtain purchase details and barcodes corresponding to a receipt, the purchase details including a first product description;
generate a first search query that includes the first product description, a list of barcodes corresponding to the barcodes, and a store identifier associated with the receipt;
execute a first search against the first dictionary using the first search query to identify a first barcode from the list of barcodes that corresponds to the first product description;
generate a second search query when the first search does not identify the first barcode, the second search query to include the first product description, the list of barcodes, and the store identifier;
execute a second search against a second dictionary based on the second search query to identify the first barcode;
associate the first barcode and the first product description; and
add the association to the first dictionary.

2. The apparatus of claim 1 , wherein the list of barcodes is a list of unique barcodes, one or more of the at least one processor circuit is to remove duplicate ones of the barcodes to generate the unique list of barcodes.

3. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to execute the first search against the first dictionary using the first search query by:
identifying first candidate barcodes in the first dictionary that correspond to the list of barcodes in the first search query;
identifying second candidate barcodes from the first candidate barcodes that correspond to the store identifier; and
identifying a first one of the second candidate barcodes, the first one of the second candidates barcodes to that include the first product description.

4. The apparatus of claim 1 , wherein the second dictionary is a products datastore that includes a plurality of products and corresponding attributes, ones of the attributes to include, at least, a respective second barcode and a respective second product description.

5. The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to execute the second search against the second dictionary using the second search query includes by:
identifying second barcodes in the second dictionary that correspond to the list of barcodes in the second search query;
comparing second product descriptions associated with the second barcodes identified in the second dictionary to the first product description in the second search query;
generating a similarity value for ones of the second product descriptions associated with the second barcodes identified in the second dictionary based on the comparison; and
selecting a candidate barcode from the list of barcodes based on the similarity value for the ones of the second product descriptions associated with the second barcodes identified in the second dictionary, the similarity value of the candidate barcode to reach a threshold value.

6. The apparatus of claim 5 , wherein the execution of the second search reveals a plurality of the second barcodes as having a respective second product description that receives a similarity score that reaches the threshold value, one or more of the at least one processor circuit is to select one of the plurality of the second barcodes that includes a similarity value with a largest value.

7. The apparatus of claim 5 , wherein one or more of the at least one processor circuit is to:
identify a match between the first product description and the candidate barcode;
generate an association between the first product description and the candidate barcode; and
add the association to the second dictionary.

8. The apparatus of claim 5 , wherein the execution of the second search does not identify a candidate barcode in the second dictionary that includes a respective second product description that receives a similarity score above the threshold value, one or more of the at least one processor circuit is to:
adjust the second search query to remove the store identifier; and
execute a third search against the second dictionary using the adjusted second search query to identify the candidate barcode from the list of barcodes that corresponds to the first product description.

9. The apparatus of claim 8 , wherein, after executing searches against the first dictionary and the second dictionary for the receipt product descriptions and the receipt barcodes corresponding to the receipt, one or more of the at least one processor circuit is to add unassociated receipt product descriptions and unassociated receipt barcodes to a list of unassociated products.

10. The apparatus of claim 9 , further including a third dictionary, wherein one or more of the at least one processor circuit is to:
generate a fourth search query that includes the unassociated receipt product descriptions from the list of unassociated products, the unassociated receipt barcodes from the list of unassociated products, and the store identifier corresponding to the receipt; and
execute a fourth search against the third dictionary using the fourth search query.

11. The apparatus of claim 10 , wherein the third dictionary is a database that includes records of previous searches corresponding to a plurality of processed receipts that did not yield associations between at least one product description and at least one barcode, ones of the records including at least unassociated product description and at least one unassociated barcode that were not matched.

12. The apparatus of claim 11 , wherein one or more of the at least one processor circuit is to execute the fourth search against the third dictionary using the fourth search query by:
comparing the unassociated receipt product descriptions and the unassociated receipt barcodes of the fourth search query to the processed receipts;
identifying a first processed receipt that includes one of the unassociated receipt product descriptions and one of the unassociated receipt barcodes;
in response to identifying the first processed receipt, identifying a match between the one of the unassociated receipt product descriptions and the one of the unassociated receipt barcodes; and
in response to not identifying the first processed receipt, adding the one of the unassociated receipt product descriptions and the one of the unassociated receipt barcodes corresponding to the receipt to as a record to a previous jobs database.

13. At least one non-transitory computer readable storage medium comprising instructions to cause at least one processor circuit to at least:
train a region detection machine learning model based on (a) images of product descriptions and barcodes in a first dictionary, (b) ground truth annotations, and (c) intersection over union (IoU) comparisons of bounding boxes associated with the images;
based on satisfying a threshold IoU value of the region detection machine learning model, obtain purchase details and barcodes corresponding to a receipt, the purchase details including a first product description;
generate a first search query that include

Assignees

• Nielsen Consumer Llc

Inventors

• Yebes Torres Jose Javier
• Sinha Aditi
• Lebrun Christine
• Oppini Fabio
• Bansal Atul
• Kumar Mukul
• Chandramouli Vignesh
• Sousa Filipa
• Mercaldi Gisella

Classifications

• G06V30/42

  based on the type of document · CPC title

• G06V30/414

  Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text · CPC title

• G06K7/1413

  1D bar codes · CPC title

• G06V30/18057

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V30/147

  Determination of region of interest · CPC title