Cart-based shopping arrangements employing probabilistic item identification

Patent metadata

Field	Value
Publication number	US-11288472-B2
Application number	US-202117142928-A
Country	US
Kind code	B2
Filing date	Jan 6, 2021
Priority date	Aug 30, 2011
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

In one aspect, a retail store has multiple sensors, including item sensors in a shopping cart for gathering data from a shopper-selected first item. At least certain of the sensor data is provided to a classifier, which was previously-trained (using data including optical data from known items) to identify possible item matches corresponding to data sensed from the first item. An item identification hypothesis that the shopper-selected first item has a particular identity is evaluated based on (a) information from the classifier, and (b) store layout data indicating items associated with a store location visited by the cart or shopper. The item identification hypothesis has a confidence score. If the score meets a criterion, an item of the hypothesized identity is added to a shopping tally. A great number of other features and arrangements are also detailed.

First claim

The invention claimed is:

1. A system comprising:
emitter devices at plural locations through a store, each emitter device emitting a locating signal distinguishable from locating signals emitted by others of the emitter devices;
a cart equipped with a sensor adapted to receive the locating signals from said emitter devices, to thereby sense position of the cart as it is moved through the store, including a visit to a first store location;
said cart further being equipped with a wireless transceiver for exchanging information with a remote computer;
a database including layout data that associates different retail items with different respective stock locations in the store, said layout data indicating retail items associated with said first store location;
plural item sensors, including one or more item sensors in said cart, said plural item sensors including first and second cameras arranged with different viewpoints and overlapping fields of view to capture imagery from a 3D item within said overlapping fields of view that is not visible to one camera alone;
a classifier that employs data including optical training data collected from known item samples; and
one or more processors with associated memory configured to evaluate a candidate identification hypothesis that a first item in the cart has a first identity, based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, and (b) information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, said hypothesis having an associated confidence score, and to add an item with said first identity to a tally associated with the cart due to said confidence score meeting a criterion.

2. The system of claim 1 in which the one or more processors are configured to evaluate multiple candidate identification hypotheses for the first item, each with an associated confidence score, and to refine said hypotheses over time as additional data is added to said ensemble.

3. The system of claim 1 in which the one or more processors with associated memory comprise a field programmable gate array.

4. The system of claim 1 in which the one or more processors with associated memory comprise an application specific circuit.

5. The system of claim 1 that includes a processor configured to virtually re-orient a patch of imagery depicting the first item captured by one of said cameras, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) information from said re-oriented patch of imagery.

6. The system of claim 1 in which one of said cart item sensors is a cart camera, and said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) barcode data sensed by the cart camera.

7. The system of claim 1 that further includes a bag positioned in an instrumented area, said instrumented area including at least one sensor adapted for collecting item data as items are moved for placement in the bag or as items rest in the bag, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, and (b) information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors including said at least one sensor included in said instrumented area.

8. The system of claim 1 in which said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) shopper history data.

9. The system of claim 1 that further includes a weight transducer positioned to monitor removal of stock from a store shelf, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) data from said weight transducer.

10. The system of claim 1 that further includes an inventory monitoring camera positioned to monitor removal of stock from a store shelf, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) data from said inventory monitoring camera.

11. The system of claim 1 in which said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) data indicating a time at which said cart visited the first store location.

12. The system of claim 1 that further includes a weight sensor in the cart and a shelf-mounted sensor, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, (c) data from said weight sensor in the cart, and (d) data collected by said shelf-mounted sensor.

13. The system of claim 1 further comprising a processor configured to derive feature vector data, including luminance gradient information, from imagery depicting said item, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) said feature vector data.

14. The system of claim 1 that further includes a 3D sensor comprising one or plural cameras, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plural item sensors, and (c) 3D shape information about said first item sensed by said 3D sensor.

15. The system of claim 1 that includes a processor that warps imagery from said first and second cameras, wherein said hypothesis is based on an ensemble of data including (a) said layout data indicating retail items associated with said first store location, (b) the information from said classifier identifying possible item matches corresponding to information sensed from the first item by said plura

Assignees

• Digimarc Corp

Inventors

• Rodriguez Tony F
• Davis Bruce L
• Rhoads Geoffrey B
• Reed Alastair M
• Evans Eric D
• Gerlach Rebecca L
• Stach John F
• Footen Marc G

Classifications

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06V30/19173

  Classification techniques · CPC title

• G06V20/63

  Scene text, e.g. street names · CPC title

• G06V10/10

  Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) · CPC title

• G06V30/224

  of printed characters having additional code marks or containing code marks · CPC title