Item placement detection and optimization in material handling systems

Patent metadata

Field	Value
Publication number	US-11392891-B2
Application number	US-202017088453-A
Country	US
Kind code	B2
Filing date	Nov 3, 2020
Priority date	Nov 3, 2020
Publication date	Jul 19, 2022
Grant date	Jul 19, 2022

Abstract

A method includes: obtaining, from an image sensor mounted on a mobile automation apparatus, an image representing a plurality of items on a support structure in a facility; responsive to detection of the items in the image, for each item: obtaining an item region defining an area of the image containing the item; obtaining a performance metric corresponding to the item; encoding the performance metric as a visual attribute; and generating an item overlay using the visual attribute; and controlling a display to present the image, and each of the item overlays placed over the corresponding item regions.

First claim

The invention claimed is:

1. A method, comprising:
obtaining, from an image sensor disposed in a facility, an image representing a plurality of items on a support structure in the facility;
responsive to detection of the items in the image, for each item:
obtaining an item region defining an area of the image containing the item;
obtaining a performance metric corresponding to the item;
encoding the performance metric as a visual attribute; and
generating an item overlay using the visual attribute; and

controlling a display to present the image and each of the item overlays placed over the corresponding item regions.

2. The method of claim 1 , wherein obtaining the performance metric includes retrieving the performance metric from a repository.

3. The method of claim 2 , wherein the performance metric includes an indication of revenue associated with the item over a time period.

4. The method of claim 1 , wherein the item region defines a maximum capacity for the item on the support structure.

5. The method of claim 4 , wherein obtaining the performance metric includes:
obtaining an occupied portion of the item region;
retrieving a previous occupied portion of the item region from a previously captured image; and
based on the occupied portion, the previous occupied portion, and a time period separating the image and the previous captured image, generating a rate of consumption for the item.

6. The method of claim 1 , further comprising:
generating, based on a comparison of the performance metrics for at least a first item and a second item, a relocation indicator defining updated placements for the first item and the second item on the support structure; and
presenting the relocation indicator with the image and the item overlays.

7. The method of claim 1 , wherein encoding the performance metric as a visual attribute includes selecting a color corresponding to the performance metric.

8. The method of claim 7 , wherein encoding the performance metric as a visual attribute includes comparing the performance metric to an upper threshold corresponding to a first color, and a lower threshold corresponding to a second color.

9. The method of claim 1 , further comprising, for each item:
determining a rate of change of the performance metric;
encoding the rate of change as an additional visual attribute; and
generating an additional item overlay using the additional visual attribute.

10. A computing device, comprising:
a communications interface, and;
a processor configured to:
obtain, from an image sensor disposed in a facility, an image representing a plurality of items on a support structure in the facility;
responsive to detection of the items in the image, for each item:
obtain an item region defining an area of the image containing the item;
obtain a performance metric corresponding to the item;
encode the performance metric as a visual attribute; and
generate an item overlay using the visual attribute; and

control a display to present the image and each of the item overlays placed over the corresponding item regions.

11. The computing device of claim 10 , wherein the processor is configured to obtain the performance metric includes by retrieving the performance metric from a repository.

12. The computing device of claim 11 , wherein the performance metric includes an indication of revenue associated with the item over a time period.

13. The computing device of claim 10 , wherein the item region defines a maximum capacity for the item on the support structure.

14. The computing device of claim 13 , wherein the processor is configured to obtain the performance metric by:
obtaining an occupied portion of the item region;
retrieving a previous occupied portion of the item region from a previously captured image; and
based on the occupied portion, the previous occupied portion, and a time period separating the image and the previous captured image, generating a rate of consumption for the item.

15. The computing device of claim 10 , wherein the processor is further configured to:
generate, based on a comparison of the performance metrics for at least a first item and a second item, a relocation indicator defining updated placements for the first item and the second item on the support structure; and
control the display to present the relocation indicator with the image and the item overlays.

16. The computing device of claim 10 , wherein the processor is configured, to encode the performance metric as a visual attribute, to select a color corresponding to the performance metric.

17. The computing device of claim 16 , wherein the processor is configured, to encode the performance metric as a visual attribute, to compare the performance metric to an upper threshold corresponding to a first color, and a lower threshold corresponding to a second color.

18. The computing device of claim 10 , wherein the processor is further configured to:
determine a rate of change of the performance metric;
encode the rate of change as an additional visual attribute; and
generate an additional item overlay using the additional visual attribute.

19. A system, comprising:
a communications interface;
an image sensor; and
a processor coupled to the communications interface and the image sensor, the processor configured to:
obtain, from the image sensor, an image representing a plurality of items on a support structure in the facility;
responsive to detection of the items in the image, for each item;
obtain an item region defining an area of the image containing the item;
obtain a performance metric corresponding to the item;
encode the performance metric as a visual attribute; and
generate an item overlay using the visual attribute; and

communicate the image and each of the item overlays placed over the corresponding item regions to a display.

20. The system of claim 19 , wherein the image sensor is a fixed position camera disposed in a facility.

21. The system of claim 19 , wherein the image sensor is disposed on a mobile automation apparatus configured to navigate the facility.

22. The system of claim 19 , wherein the processor determines an occupancy trend of the item on the support structure as the performance metric based on the image from the image sensor and encodes the occupancy trend as the visual attribute.

23. The system of claim 19 , wherein the processor causes the display to display a heat map of the item overlays on the support structure.

24. The system of claim 23 , wherein the heat map is one of a color heat map and a pattern heat map.

Assignees

• Zebra Tech Corp

Inventors

• Stagg Adrian J
• Perrella Marco
• Varley Jordan K
• Kennedy Patrick

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G09G5/377

  for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence) · CPC title

• G06Q10/087 Primary

  Inventory or stock management, e.g. order filling, procurement or balancing against orders · CPC title

• G05D1/0094

  involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target · CPC title