Method, system and apparatus for gap detection in support structures with peg regions

Patent metadata

Field	Value
Publication number	US-11080566-B2
Application number	US-201916429687-A
Country	US
Kind code	B2
Filing date	Jun 3, 2019
Priority date	Jun 3, 2019
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

A method of detecting gaps on a support structure includes: obtaining, at an imaging controller, (i) a plurality of depth measurements representing the support structure according to a common frame of reference, and (ii) a plurality of label indicators each defining a label position in the common frame of reference; for each of the label indicators: classifying the label indicator as either a peg label or a shelf label, based on a portion of the depth measurements selected according to the label position and a portion of the depth measurements adjacent to the label position; generating an item search space in the common frame of reference according to the class of the label indicator; and determining, based on a subset of the depth measurements within the item search space, whether the item search space contains an item.

First claim

The invention claimed is:

1. A method of detecting gaps on a support structure, the method comprising:
obtaining, at an imaging controller, (i) a plurality of depth measurements representing the support structure according to a common frame of reference, and (ii) a plurality of label indicators each defining a boundary of a label mounted on either a peg or a shelf edge of the support structure in the common frame of reference;
for each of the label indicators:
assigning, to the label indicator, either a peg-mounted label class or a shelf edge-mounted label class, based on a first portion of the depth measurements within the boundary, and a second portion of the depth measurements outside the boundary and adjacent to the boundary;
generating an item search space in the common frame of reference according to the class of the label indicator; and
determining, based on a subset of the depth measurements within the item search space, whether the item search space contains an item.

2. The method of claim 1 , further comprising: when the determination is negative, generating a gap indicator in association with the label indicator.

3. The method of claim 1 , wherein assigning a class to the label indicators comprises:
selecting, as the second portion of the depth measurements, neighbor windows on either side of the label boundary;
generating depth metrics for each of the first portion of the depth measurements and the neighbor windows;
when error measurements between the first portion depth metric and each of the neighbor window depth metrics exceed a threshold, classifying the label indicator as a peg label; and
otherwise, classifying the label indicator as a shelf label.

4. The method of claim 1 , wherein classifying the label indicators comprises:
identifying a pair of adjacent label indicators separated by a distance below a distance threshold;
selecting, as the first portion of the depth measurements, (i) label windows corresponding to the interiors of the boundaries of each of the pair, (ii) neighbor windows on either side of the boundaries of the pair, and (iii) a gap window between the boundaries of the pair;
generating a single depth metric for the label windows;
generating depth metrics for each of the neighbor windows and the gap window;
when error measurements between the single depth metric and each of the neighbor window depth metric and the gap widow depth metric exceed a threshold, classifying the label indicator as a peg label; and
otherwise, classifying the label indicator as a shelf label.

5. The method of claim 3 , wherein the depth metric is a vector containing at least one of: a number of depth measurements, a minimum depth, a maximum depth, a mean depth, a first quartile depth, a third quartile depth, a median depth, a depth range, and a standard deviation.

6. The method of claim 5 , wherein the error measurements are root-mean square errors (RMSEs).

7. The method of claim 1 , wherein generating the item search space includes, when the label indicator us classified as a peg label:
setting side boundaries based on left and right neighbor label indicators;
setting an upper boundary below the label indicator; and
setting a lower boundary according to a class of a lower neighbor label indicator.

8. The method of claim 1 , wherein generating the item search space includes, when the label indicator is classified as a shelf label:
setting side boundaries based on the label indicator and a neighbor label indicator;
setting an upper boundary according to a class of an upper neighbor label indicator; and
setting a lower boundary based on the label indicator.

9. The method of claim 1 , wherein determining whether the item search space contains an item comprises:
generating a depth metric from the depth measurements within the item search space;
when the metric meets a threshold, determining that the item search space contains an item; and
when the metric does not meet a threshold, determining that the item search space contains a gap.

10. The method of claim 1 , wherein determining whether the item search space contains an item comprises:
generating a depth map from the depth measurements within the item search space;
generating a set of binary masks from the depth map; and
detecting the presence or absence of an item based on the binary masks.

11. A computing device comprising:
a memory storing (i) a plurality of depth measurements representing a support structure according to a common frame of reference, and (ii) a plurality of label indicators each defining a boundary of a label position mounted on either a peg or a shelf edge of the support structure in the common frame of reference;
an imaging controller configured, for each of the label indicators, to:
assign, to the label indicator, either a peg-mounted label class or a shelf edge-mounted label class, based on a first portion of the depth measurements within the boundary, and a second portion of the depth measurements outside the boundary and adjacent to the boundary;
generate an item search space in the common frame of reference according to the class of the label indicator; and
determine, based on a subset of the depth measurements within the item search space, whether the item search space contains an item.

12. The computing device of claim 11 , wherein the imaging controller is further configured to: when the determination is negative, generate a gap indicator in association with the label indicator.

13. The computing device of claim 11 , wherein the imaging controller is further configured, in order to classify assign a class to the label indicators, to:
select, as the second portion of the depth measurements, neighbor windows on either side of the label boundary;
generate depth metrics for each of the first portion of the depth measurements and the neighbor windows;
when error measurements between the first portion depth metric and each of the neighbor window depth metrics exceed a threshold, classify the label indicator as a peg label; and
otherwise, classify the label indicator as a shelf label.

14. The computing device of claim 11 , wherein the imaging controller is further configured, in order to classify the label indicators, to:
identify a pair of adjacent label indicators separated by a distance below a distance threshold;
select, as the first portion of the depth measurements, (i) label windows corresponding to the interiors of the boundaries of each of the pair, (ii) neighbor windows on either side of the boundaries of the pair, and (iii) a gap window between the boundaries of the pair;
generate a single depth metric for the label windows;
generate depth metrics for each of the neighbor windows and the gap window;
when error measurements between the single depth metric and each of the neighbor window depth metric and the gap widow depth metric exceed a threshold, classify the label indicator as a peg label; and
otherwise, classify the label indicator as a shelf label.

15. The computing device of claim 13 , wherein the depth metric is a vector containing at least one of a number of depth measurements, a minimum depth, a maximum depth, a mean depth, a first quartile depth, a third quartile depth, a median depth, a depth range, and a standard deviation.

16. The computing device of claim 15 , wherein the error measurements are root-mean square errors (RMSEs).

17. The computing device of claim 11 , wherein the imaging controller is further configured, in order to generate the item search space, when the label indicator us classified as a peg

Assignees

• Zebra Tech Corp

Inventors

• Ghazel Mohsen
• Lam Joseph
• Gan Qifeng
• Lomakin Kiril

Classifications

• G06V10/255

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06F18/2433

  Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T7/50

  Depth or shape recovery · CPC title