Identifying item barcodes

Patent metadata

Field	Value
Publication number	US-11328147-B1
Application number	US-202117213031-A
Country	US
Kind code	B1
Filing date	Mar 25, 2021
Priority date	Jun 27, 2019
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

This disclosure describes using trained classifiers and additional data to identify items placed into a tote. The techniques may use a first classifier on image data representing an item to determine a region associated with a barcode of the item and may use one or more second classifiers to analyze the region for determining a candidate barcode. The techniques may then determine whether the candidate barcode corresponds to the actual barcode of the item. If so, the techniques may update a virtual cart associated with a user account to indicate the acquisition of the item. If not, then the techniques may use the additional data for identifying the item. For example, the techniques determine a current location of a tote for determining items that are within a threshold distance of the location and may use this information, along with the candidate barcode, to identify the item placed into the tote.

First claim

What is claimed is:

1. A method comprising:
receiving first image data from a first camera mounted to a shopping cart;
analyzing the first image data to identify a user profile;
receiving second image data from a second camera mounted to the shopping cart, the second image data representing an item, wherein the second image data includes identifying information printed on the item;
analyzing the second image data to identify first data corresponding to the identifying information;
receiving second data for identifying the item, the second data based at least in part on the user profile; and
determining an item identifier corresponding to the item based at least in part on the first data and the second data.

2. The method as recited in claim 1 , wherein:
the identifying information comprises a barcode;
the first data comprises a candidate character string corresponding to the barcode; and
the determining comprises determining the item identifier corresponding to the item based at least in part on the candidate character string and the second data.

3. The method as recited in claim 1 , wherein the analyzing comprises:
analyzing the second image data to identify a region of the second image data corresponding to the identifying information; and
analyzing the region of the second image data to identify the first data corresponding to the identifying information.

4. The method as recited in claim 3 , wherein:
the analyzing the second image data to identify the region of the second image data comprises analyzing the second image data using a first trained classifier; and
the analyzing the region of the second image data to identify the first data corresponding to the identifying information comprises analyzing the region of the second image data using one or more second trained classifiers.

5. The method as recited in claim 1 , wherein the the second camera generates the second image data as the item is placed into the shopping cart.

6. The method as recited in claim 1 , wherein the receiving the second data comprises receiving an indication of current contents of a virtual cart associated with the user profile.

7. The method as recited in claim 1 , further comprising:
determining a confidence level that the item identifier corresponds to the item based at least in part on: (i) a distance between an inventory location associated with the item identifier in the facility and the first location, and (ii) a similarity between the item identifier and the first data.

8. The method as recited in claim 1 , wherein the receiving the second data comprises receiving an indication of one or more items on a list associated with the user profile.

9. The method as recited in claim 1 , wherein the receiving the second data comprises receiving additional data indicating at least one of a current time of day or a previous item acquired by a user associated with the user profile.

10. The method as recited in claim 1 , wherein the item identifier comprises a first item identifier, and further comprising:
determining a first set of candidate item identifiers using the first data, the first set of candidate item identifiers including the first item identifier; and
determining a second set of candidate item identifiers using the second data, the second set of candidate item identifiers including the first item identifier;
and wherein the determining the item identifier comprises determining the item identifier corresponding to the item based at least in part on the first set of candidate item identifiers and the second set of candidate item identifiers.

11. A shopping cart, comprising:
a first camera mounted to the shopping cart;
a second camera mounted to the shopping cart;
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising:
receiving first image data from the first camera;
analyzing the first image data to identify a user profile;
receiving second image data from the second camera, the second image data representing an item, wherein the second image data includes identifying information printed on the item;
analyzing the second image data to identify first data corresponding to the identifying information;
receiving second data for identifying the item, the second data based at least in part on the user profile; and
determining an item identifier corresponding to the item based at least in part on the first data and the second data.

12. The shopping cart as recited in claim 11 , wherein:
the identifying information comprises a barcode;
the first data comprises a candidate character string corresponding to the barcode; and
the determining comprises determining the item identifier corresponding to the item based at least in part on the candidate character string and the second data.

13. The shopping cart as recited in claim 11 , wherein the analyzing comprises:
analyzing the second image data to identify a region of the second image data corresponding to the identifying information; and
analyzing the region of the second image data to identify the first data corresponding to the identifying information.

14. The shopping cart as recited in claim 13 , wherein:
the analyzing the second image data to identify the region of the second image data comprises analyzing the second image data using a first trained classifier; and
the analyzing the region of the second image data to identify the first data corresponding to the identifying information comprises analyzing the region of the second image data using one or more second trained classifiers.

15. The shopping cart as recited in claim 11 , wherein the second camera generates the second image data as the item is placed into the shopping cart.

16. The shopping cart as recited in claim 11 , wherein the receiving the second data comprises receiving an indication of current contents of a virtual cart associated with the user profile.

17. The shopping cart as recited in claim 11 , wherein:
the one or more computer-readable non-transitory computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising determining a confidence level that the item identifier corresponds to the item based at least in part on: (i) a distance between an inventory location associated with the item identifier in the facility and the first location, and (ii) a similarity between the item identifier and the first data.

18. The shopping cart as recited in claim 11 , wherein the receiving the second data comprises receiving an indication of one or more items on a list associated with the user profile.

19. The shopping cart as recited in claim 11 , wherein the receiving the second data comprises receiving additional data indicating at least one of a current time of day or a previous item acquired by a user associated with the user profile.

20. The shopping cart as recited in claim 11 , wherein the item identifier comprises a first item identifier, and the one or more non-transitory computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising:
determining a first set of candidate item identifiers using the first data, the first set of candidate item identifiers including the first item identifier; and
determining a second set of candidate item identifiers

Assignees

• Amazon Tech Inc

Inventors

• Ahmed Ejaz
• Maron Oded
• De Bonet Jeremy S
• Ron Daniel
• Beygiharchegani Sajjad

Classifications

• G06K7/1413 Primary

  1D bar codes · CPC title

• G06V30/224

  of printed characters having additional code marks or containing code marks · CPC title

• G06V10/225

  based on a marking or identifier characterising the area · CPC title

• G06V30/19147

  Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title