Counting stacked inventory using image analysis

Patent metadata

Field	Value
Publication number	US-10169677-B1
Application number	US-201414578009-A
Country	US
Kind code	B1
Filing date	Dec 19, 2014
Priority date	Dec 19, 2014
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

Described is a system for counting stacked items using image analysis. In one implementation, an image of an inventory location with stacked items is obtained and processed to determine the number of items stacked at the inventory location. In some instances, the item closest to the camera that obtains the image may be the only item viewable in the image. Using image analysis, such as depth mapping or Histogram of Oriented Gradients (HOG) algorithms, the distance of the item from the camera and the shelf of the inventory location can be determined. Using this information, and known dimension information for the item, a count of the number of items stacked at an inventory location may be determined.

First claim

What is claimed is:

1. A computing system, comprising:
a processor; and
a memory coupled to the processor and storing program instructions that when executed by the processor causes the processor to at least:
receive from a first camera a first image of an inventory location, wherein the first image includes a representation of a plurality of inventory items vertically stacked and located at the inventory location;
determine from an inventory location data store, an item type corresponding to the inventory location;
select a first histogram of oriented gradients (“HOG”) model and a second HOG model, wherein:
the first HOG model corresponds to the item type and is representative of a first vertical stack of a first quantity of an item of the item type;
the second HOG model corresponds to the item type and is representative of a second vertical stack of a second quantity of the item of the item type, wherein the first quantity and the second quantity are different;

process the first image to generate an inventory item feature vector representative of the plurality of inventory items vertically stacked and represented in the first image;
compare the inventory item feature vector with the first HOG model and the second HOG model;
determine that the inventory item feature vector substantially matches the first HOG model; and
in response to a determination that the inventory item feature vector substantially matches the first HOG model, determine an inventory item count representative of a number of inventory items vertically stacked at the inventory location to be the first quantity.

2. The computing system of claim 1 , wherein the program instructions, that when executed by the processor to cause the processor to determine the inventory item count, further include instructions that when executed by the processor cause the processor to at least:
determine a first distance between the first camera and a first item of the plurality of inventory items vertically stacked and located at the inventory location;
determine a second distance between the first camera and a shelf of the inventory location;
determine a dimension of an inventory item of the plurality of inventory items; and
compute the inventory item count based at least in part on a difference between the first distance and the second distance and further based at least in part on the dimension of the inventory item.

3. The computing system of claim 1 , wherein the inventory item count identifies an increase or a decrease in inventory at the inventory location.

4. The computing system of claim 1 , wherein the program instructions, that when executed by the processor to cause the processor to determine the inventory item count, further include instructions that when executed by the processor cause the processor to at least:
determine a defined distance between a front edge of an inventory shelf of the inventory location and the plurality of inventory items;
determine a depth of the inventory shelf from the front edge of the inventory shelf to a back of the inventory shelf;
determine a dimension of an inventory item of the plurality of inventory items; and
compute an inventory item count based at least in part on a difference between the defined distance and the depth of the inventory shelf and further based at least in part on the dimension of the inventory item.

5. A computer-implemented method for counting stacked items, comprising:
under control of one or more computing systems configured with executable instructions,
receiving from a camera an image of a plurality of stacked items positioned at an inventory location;
determining from the image, and based at least in part on a histogram of oriented gradients (HOG) model determined to correspond to the image, a first distance between an item of the plurality of stacked items and the camera, wherein the HOG model is representative of at least one of the plurality of stacked items, or the item;
determining an item dimension representative of at least one of a height, a width, or a length of the item; and
computing an item count based at least in part on:
the first distance between the item and the camera; and
the item dimension.

6. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a shelf of the inventory location and the camera; and
wherein the item count is further based at least in part on:
a difference between the first distance between the item and the camera and the second distance; and
the item dimension.

7. The computer-implemented method of claim 6 , wherein:
the second distance is maintained in an inventory location data store; and
the first distance is further determined based at least in part on depth information received with the image.

8. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a front edge of a shelf of the inventory location and the item;
determining a depth of the shelf from the front edge of the shelf to a back of the shelf; and
computing the item count based at least in part on:
a difference between the second distance between the front edge of the shelf and the item,
the depth of the shelf; and
the item dimension of the item.

9. The computer-implemented method of claim 5 , wherein the first distance is further determined based at least in part on a comparison of the image with a plurality of images obtained by the camera to obtain depth information for the item.

10. The computer-implemented method of claim 5 , further comprising:
determining a second distance between a shelf of the inventory location and the camera;
determining a third distance between a front edge of the shelf and the item;
determining a fourth distance between the front edge of the shelf and a back of the shelf;
computing the item count based at least in part on:
a first quotient of:
a difference between the first distance and the second distance; and
the item dimension; and

a second quotient of:
a difference between the third distance and the fourth distance; and
a second item dimension.

11. The computer-implemented method of claim 5 , wherein the camera is mounted to an underneath side of a shelf positioned above the inventory location and oriented downward toward the inventory location.

12. The computer-implemented method of claim 5 , wherein the item count identifies at least one of a number of items stacked vertically at the inventory location or a number of items stacked horizontally at the inventory location.

13. A computer-implemented method, comprising:
under control of one or more computing systems configured with executable instructions,
receiving from a camera an image of a plurality of stacked items positioned at an inventory location;
determining from the image a first distance between an item of the plurality of stacked items and the camera wherein:
the first distance is determined based at least in part on a change in a magnification level of the item represented in the image when the item is compared with a histogram of oriented gradients model (“HOG”); and
the magnification level corresponds with the first distance between the item and the camera;

determining an item dimension representative of at least one of a height, a width, or a length of the item; and
computing an item count based at least in part on:
the first distance between the item and the camera; and
the item dimension.

14. The computer-implemented method of claim 13 , further comprising:
determining a second distance between a shelf of the inventory loca

Assignees

• Amazon Tech Inc

Inventors

• Ren Xiaofeng
• Bo Liefeng
• Manyam Ohil Krishnamurthy
• MISRA Avishkar
• Raghavan Sudarshan Narasimha
• Towers Christopher Robert
• Gopal Gopi Prashanth
• Asmi Yasser Baseer

Classifications

• H04W4/33

  for indoor environments, e.g. buildings · CPC title

• H04W4/021

  Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences · CPC title

• H04W4/80

  Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication · CPC title

• G06Q10/087

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title