Deep learning-based store realograms

Patent metadata

Field	Value
Publication number	US-11295270-B2
Application number	US-201916679035-A
Country	US
Kind code	B2
Filing date	Nov 8, 2019
Priority date	Aug 7, 2017
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

Systems and techniques are provided for tracking inventory items in an area of real space. A plurality of cameras, or other sensors, produce respective sequences of images in corresponding fields of view in the real space. The field of view of each camera overlaps with the field of view of at least one other camera. The system is coupled to the plurality of cameras and uses the sequences of images produced by at least two cameras in the plurality of cameras to identify inventory events. The inventory event includes an item identifier, a location and a timestamp. A plurality of cells having coordinates in the area of real space are stored as a data set in the memory. The processing system calculates scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

First claim

What is claimed is:

1. A system for tracking inventory items in an area of real space, comprising:
a plurality of sensors, sensors in the plurality of sensors producing respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors; and
a processing system coupled to the plurality of sensors, the processing system dividing the area of real space into a plurality of cells having volume and coordinates in the real space, and including logic that uses the sequences of images produced by at least two sensors having overlapping fields of view in the plurality of sensors to
identify inventory items and their locations in three dimensions,
identify inventory events in which locations of identified inventory items are changed,
in response to the identified inventory events, match locations in three dimensions of identified inventory items with cells in the plurality of cells, and
cause to display on a display screen (i) an identification of the plurality of cells that includes at least a first cell and a second cell, and (ii) indication of a first count of locations of a first type of inventory items matching the first cell and indication of a second count of locations of the first type of inventory items matching the second cell.

2. The system of claim 1 , wherein the inventory events include an item identifier, a put or take indicator, locations of inventory items represented by positions in three dimensions of the area of real space, and a timestamp.

3. The system of claim 1 , including logic to maintain a data set representing inventory items matched with cells in the plurality of cells.

4. The system of claim 1 , wherein the area of real space includes a plurality of inventory locations, and including logic to match inventory locations with cells in the plurality of cells.

5. A system for tracking inventory items in an area of real space, comprising:
a plurality of sensors, sensors in the plurality of sensors producing respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors; and
a processing system coupled to the plurality of sensors, the processing system dividing the area of real space into a plurality of cells having volume and coordinates in the real space, and including logic that uses the sequences of images produced by at least two sensors in the plurality of sensors to identify inventory events in which locations of inventory items are changed, and in response to the inventory events, and to match locations of inventory items with cells in the plurality of cells, the processing system including logic that calculates scores at a scoring time for inventory items having locations matching particular cells, the scores based on counts of inventory events.

6. The system of claim 5 , the logic that calculates scores for cells uses sums of puts and takes of inventory items weighted by a separation between timestamps of the puts and takes and the scoring time.

7. The system of claim 5 , the processing system including logic to render a display image representing cells in the plurality of cells and the scores for the cells.

8. The system of claim 5 , the processing system including logic to select a set of inventory items for each cell based on the scores.

9. The system of claim 1 , wherein the area of real space includes a plurality of inventory locations having coordinates in the real space, and including
a planogram in memory specifying a distribution of inventory items in inventory locations in the area of real space, and
logic to maintain the data representing inventory items matched with cells in the plurality of cells, and logic to determine misplaced items by comparing the data representing inventory items matched with cells to the distribution of inventory items in the inventory locations specified in the planogram.

10. The system of claim 1 , wherein the area of real space includes a plurality of inventory locations having coordinates in the area of real space, and including logic to maintain the data representing inventory items matched with cells in the plurality of cells, and logic that determines, for a particular inventory item matched with a particular cell, whether a count of the particular inventory item in that cell is below a threshold for re-stocking the particular inventory item on an inventory location correlating with the particular cell.

11. A method for tracking inventory items in an area of real space, the method including:
using a plurality of sensors to produce respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors;
identifying inventory items and their locations in three dimensions, and inventory events using the sequences of images produced by at least two sensors having overlapping fields of view in the plurality of sensors in which locations of the identified inventory items are changed;
using the identified inventory events to match locations in three dimensions of identified inventory items with cells in a plurality of cells in the area of real space having volume and coordinates in the real space; and
causing to display on a display screen (i) an identification of the plurality of cells that includes at least a first cell and a second cell, and (ii) indication of a first count of locations of a first type of inventory items matching the first cell and indication of a second count of locations of the first type of inventory items matching the second cell.

12. The method of claim 11 , wherein the inventory events include an item identifier, a put or take indicator, locations represented by positions along three axes of the area of real space, and a timestamp.

13. The method of claim 11 , further including maintaining a data set representing inventory items matched with cells in the plurality of cells.

14. The method of claim 13 , wherein the area of real space includes a plurality of inventory locations having coordinates in the real space, and including matching inventory locations with cells in the plurality of cells.

15. A method for tracking inventory items in an area of real space, the method including:
using a plurality of sensors to produce respective sequences of images of corresponding fields of view in the real space, the field of view of each sensor overlapping with the field of view of at least one other sensor in the plurality of sensors;
identifying inventory events using the sequences of images produced by at least two sensors in the plurality of sensors in which locations of inventory items are changed, and
using the inventory events to match locations of inventory items with cells in a plurality of cells in the area of real space having volume and coordinates in the real space, further including calculating scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

16. The method of claim 15 , further including calculating scores for cells using sums of puts and takes of inventory items weighted by a separation between timestamps of the puts and takes and the scoring time, and storing the scores in a memory.

17. The method of claim 15 , further including rendering a display image representing cells in the plural

Assignees

• Standard Cognition Corp

Inventors

• Fisher Jordan E
• Fischetti Daniel L
• Suswal Michael S
• Locascio Nicholas J

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/10004

  Still image; Photographic image · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title