Method, system and apparatus for correcting translucency artifacts in data representing a support structure

Patent metadata

Field	Value
Publication number	US-10740911-B2
Application number	US-201815946165-A
Country	US
Kind code	B2
Filing date	Apr 5, 2018
Priority date	Apr 5, 2018
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

A method in an imaging controller of correcting translucency artifacts in data representing one or more objects disposed on a shelf includes: obtaining a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf; obtaining (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge; generating a depth map containing, for each of a plurality of positions in the plane, a nearest object depth; detecting an upper object boundary in the depth map between the upper and lower support surface edges; updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and storing the corrected depth map.

First claim

The invention claimed is:

1. A method in an imaging controller of correcting translucency artifacts in data representing one or more objects disposed on a shelf, comprising:
obtaining a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf;
obtaining (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge;
generating a depth map containing, for each of a plurality of positions in the plane, a nearest object depth;
detecting an upper object boundary in the depth map between the upper and lower shelf edges;
generating a corrected depth map by updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and
storing the corrected depth map.

2. The method of claim 1 , further comprising presenting the corrected depth map to a gap detector for use in detecting gaps on the shelf.

3. The method of claim 1 , further comprising:
prior to generating the depth map, aligning the depth measurements with a frame of reference based on the plane.

4. The method of claim 1 , wherein generating the depth map includes generating each nearest object depth by:
assigning each depth measurement to one of a plurality of bins arranged in a three-dimensional grid, to generate a count of depth measurements falling within each bin;
for each of the plurality of positions in the plane, traversing a subset of the bins in a direction perpendicular to the plane and accumulating the respective counts of the subset of bins until the accumulated counts reach a threshold; and
setting the nearest object depth as a depth of a final one of the subset of bins traversed.

5. The method of claim 1 , wherein detecting the upper object boundary comprises:
beginning at the location of the upper shelf edge, traversing a strip of the depth map from the location of the upper shelf edge toward the location of the lower shelf edge; and
determining whether a change in depth between traversed positions in the strip exceeds a predefined threshold.

6. The method of claim 5 , wherein the predefined threshold defines a decrease in depth.

7. The method of claim 5 , wherein generating the corrected depth map comprises setting the nearest object depths of each position in the strip between the upper object boundary and the location of the lower shelf edge to the depth of the upper object boundary.

8. The method of claim 5 , wherein the strip is a line.

9. The method of claim 5 , wherein the strip has a predefined width greater than one nearest object depth value.

10. The method of claim 1 , further comprising:
prior to generating the corrected depth map, correcting null values in the depth map by performing a leaky convolution on the depth map.

11. A computing device for correcting translucency artifacts in data representing one or more objects disposed on a shelf, comprising:
a memory; and
an imaging controller connected to the memory, the imaging controller including:
a preprocessor configured to obtain a plurality of depth measurements captured by a depth sensor and corresponding to an area containing the shelf;
the preprocessor further configured to obtain (i) a definition of a plane containing edges of the shelf, (ii) a location in the plane of an upper shelf edge, and (iii) a location in the plane of a lower shelf edge adjacent to the upper shelf edge;
a map generator configured to generate a depth map containing, for each of a plurality of positions in the plane, a nearest object depth;
a corrector configured to detect an upper object boundary in the depth map between the upper and lower shelf edges; and
the corrector further configured to generate a corrected depth map by updating each nearest object depth between the upper object boundary and the lower shelf edge to contain a depth of the upper object boundary; and
the imaging controller further configured to store the corrected depth map in the memory.

12. The computing device of claim 11 , wherein the imaging controller is further configured to present the corrected depth map to a gap detector for use in detecting gaps on the shelf.

13. The computing device of claim 11 , wherein the preprocessor is further configured, prior to generation of the depth map, to aligning the depth measurements with a frame of reference based on the plane.

14. The computing device of claim 11 , wherein the map generator is configured to generate each nearest object depth by:
assigning each depth measurement to one of a plurality of bins arranged in a three-dimensional grid, to generate a count of depth measurements falling within each bin;
for each of the plurality of positions in the plane, traversing a subset of the bins in a direction perpendicular to the plane and accumulating the respective counts of the subset of bins until the accumulated counts reach a threshold; and
setting the nearest object depth as a depth of a final one of the subset of bins traversed.

15. The computing device of claim 11 , wherein the corrector is further configured to detect the upper object boundary by:
beginning at the location of the upper shelf edge, traversing a strip of the depth map from the location of the upper shelf edge toward the location of the lower shelf edge; and
determining whether a change in depth between traversed positions in the strip exceeds a predefined threshold.

16. The computing device of claim 15 , wherein the predefined threshold defines a decrease in depth.

17. The computing device of claim 15 , wherein the corrector is further configured to generate the corrected depth map by setting the nearest object depths of each position in the strip between the upper object boundary and the location of the lower shelf edge to the depth of the upper object boundary.

18. The computing device of claim 15 , wherein the strip is a line.

19. The computing device of claim 15 , wherein the strip has a predefined width greater than one nearest object depth value.

20. The computing device of claim 11 , wherein the map generator is further configured to:
prior to generating the corrected depth map, correct null values in the depth map by performing a leaky convolution on the depth map.

Assignees

• Symbol Technologies Llc

Inventors

• Phan Raymond
• Rzeszutek Richard Jeffrey
• Segall Iaacov Coby

Classifications

• G06V20/68

  Food, e.g. fruit or vegetables · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06T7/50 Primary

  Depth or shape recovery · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title