Method, system and apparatus for support structure depth determination

Patent metadata

Field	Value
Publication number	US-11506483-B2
Application number	US-201816152986-A
Country	US
Kind code	B2
Filing date	Oct 5, 2018
Priority date	Oct 5, 2018
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A method of determining a support structure depth of a support structure having a front and a back separated by the support structure depth includes: obtaining a point cloud of the support structure, and a mask indicating, for a plurality of portions of an image of the support structure captured from a capture pose, respective confidence levels that the portions depict the back of the support structure; selecting, from the point cloud, an initial set of points located within a field of view originating at the capture pose; selecting, from the initial set of points, an unoccluded subset of depth measurements, the depth measurements in the unoccluded subset corresponding to respective image coordinates; retrieving, from the mask, a confidence level for each of the depth measurements in the unoccluded subset; and based on the depth measurements in the unoccluded subset and the retrieved confidence levels, determining the support structure depth.

First claim

The invention claimed is:

1. A method of determining a support structure depth of a support structure having a front and a back separated by the support structure depth, the method comprising:
obtaining (i) a point cloud of the support structure, and (ii) a mask indicating, for a plurality of portions of an image of the support structure captured from a capture pose, respective confidence levels that the portions depict the back of the support structure;
selecting, from the point cloud, an initial set of points located within a field of view originating at the capture pose;
selecting, from the initial set of points, an unoccluded subset of depth measurements, the depth measurements in the unoccluded subset corresponding to respective image coordinates;
retrieving, from the mask, a confidence level for each of the depth measurements in the unoccluded subset; and
based on the depth measurements in the unoccluded subset and the retrieved confidence levels, determining the support structure depth.

2. The method of claim 1 , further comprising:
obtaining a further mask corresponding to a further image of the support structure captured from a further capture pose;
selecting a further initial set of points;
selecting a further unoccluded set of depth measurements;
retrieving a further confidence level for each of the depth measurements in the further unoccluded subset.

3. The method of claim 2 , further comprising:
determining the support structure depth based on the depth measurements in the unoccluded subset, the depth measurements in the further unoccluded subset, the confidence levels and the further confidence levels.

4. The method of claim 1 , wherein a capture pose defines a camera position and orientation within a common frame of reference.

5. The method of claim 1 , wherein selecting the initial set of points comprises:
generating a tree data structure containing, for each point in the point cloud, first and second dimensions orthogonal to the support structure depth;
determining a field of view center in the first and second dimensions; and
retrieving, from the tree data structure, the initial set of points within a predefined radius of the field of view center.

6. The method of claim 1 , wherein selecting the unoccluded subset of depth measurements comprises:
determining image coordinates corresponding to each of the initial set of points;
identifying neighbor groups of the image coordinates; and
for each neighbor group, selecting the image coordinate corresponding to the smallest depth measurement.

7. The method of claim 1 , wherein determining the support structure depth comprises:
obtaining a plane definition corresponding to the front of the support structure;
transforming each depth measurement of the unoccluded subset of depth measurements to a depth relative to the plane definition;
weighting each transformed depth measurement according to the retrieved confidence levels; and
determining an average of the weighted depth measurements.

8. The method of claim 1 , further comprising:
prior to determining the support structure depth, discarding depth measurements for which the retrieved confidence levels are below a minimum confidence threshold.

9. The method of claim 1 , further comprising:
prior to determining the support structure depth, discarding depth measurements exceeding a maximum depth threshold.

10. A computing device for determining a support structure depth of a support structure having a front and a back separated by the support structure depth, the computing device comprising:
a memory storing (i) a point cloud of the support structure, and (ii) a mask indicating, for a plurality of portions of an image of the support structure captured from a capture pose, respective confidence levels that the portions depict the back of the support structure;
an imaging controller connected to the memory and configured to:
select, from the point cloud, an initial set of points located within a field of view originating at the capture pose;
select, from the initial set of points, an unoccluded subset of depth measurements, the depth measurements in the unoccluded subset corresponding to respective image coordinates;
retrieve, from the mask, a confidence level for each of the depth measurements in the unoccluded subset; and
based on the depth measurements in the unoccluded subset and the retrieved confidence levels, determine the support structure depth.

11. The computing device of claim 10 , wherein the imaging controller is further configured to:
obtain a further mask corresponding to a further image of the support structure captured from a further capture pose;
select a further initial set of points;
select a further unoccluded set of depth measurements;
retrieve a further confidence level for each of the depth measurements in the further unoccluded subset.

12. The computing device of claim 11 , wherein the imaging controller is further configured to:
determine the support structure depth based on the depth measurements in the unoccluded subset, the depth measurements in the further unoccluded subset, the confidence levels and the further confidence levels.

13. The computing device of claim 10 , wherein a capture pose defines a camera position and orientation within a common frame of reference.

14. The computing device of claim 10 , wherein the imaging controller is further configured, to select the initial set of points, to:
generate a tree data structure containing, for each point in the point cloud, first and second dimensions orthogonal to the support structure depth;
determine a field of view center in the first and second dimensions; and
retrieve, from the tree data structure, the initial set of points within a predefined radius of the field of view center.

15. The computing device of claim 10 , wherein the imaging controller is further configured, to select the unoccluded subset of depth measurements, to:
determine image coordinates corresponding to each of the initial set of points;
identify neighbor groups of the image coordinates; and
for each neighbor group, select the image coordinate corresponding to the smallest depth measurement.

16. The computing device of claim 10 , wherein the imaging controller is further configured, to determine the support structure depth, to:
obtain a plane definition corresponding to the front of the support structure;
transform each depth measurement of the unoccluded subset of depth measurements to a depth relative to the plane definition;
weight each transformed depth measurement according to the retrieved confidence levels; and
determine an average of the weighted depth measurements.

17. The computing device of claim 10 , wherein the imaging controller is further configured to:
prior to determining the support structure depth, discard depth measurements for which the retrieved confidence levels are below a minimum confidence threshold.

18. The computing device of claim 10 , wherein the imaging controller is further configured to:
prior to determining the support structure depth, discard depth measurements exceeding a maximum depth threshold.

19. A computer-readable medium having stored thereon computer-executable instructions, the instructions comprising:
obtaining (i) a point cloud of the support structure, and (ii) a mask indicating, for a plurality of portions of an image of the support structure captured from a capture pose, respective confidence levels that the portions depict the back of the support stru

Assignees

• Zebra Tech Corp

Inventors

• Phan Raymond
• Yu Yuanhao
• Rzeszutek Richard Jeffrey
• Lam Joseph

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/0008

  checking presence/absence · CPC title

• G06T7/62

  of area, perimeter, diameter or volume · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G01B11/22 Primary

  for measuring depth · CPC title