System and method for simultaneous consideration of edges and normals in image features by a vision system

Patent metadata

Field	Value
Publication number	US-10957072-B2
Application number	US-201815901117-A
Country	US
Kind code	B2
Filing date	Feb 21, 2018
Priority date	Feb 21, 2018
Publication date	Mar 23, 2021
Grant date	Mar 23, 2021

Abstract

This invention applies dynamic weighting between a point-to-plane and point-to-edge metric on a per-edge basis in an acquired image using a vision system. This allows an applied ICP technique to be significantly more robust to a variety of object geometries and/or occlusions. A system and method herein provides an energy function that is minimized to generate candidate 3D poses for use in alignment of runtime 3D image data of an object with model 3D image data. Since normals are much more accurate than edges, the use of normal is desirable when possible. However, in some use cases, such as a plane, edges provide information in relative directions the normals do not. Hence the system and method defines a “normal information matrix”, which represents the directions in which sufficient information is present. Performing (e.g.) a principal component analysis (PCA) on this matrix provides a basis for the available information.

First claim

What is claimed is:

1. A method for finding the pose of a 3D model in a 3D image of an object acquired by a 3D camera assembly, comprising the steps of:
providing the 3D model to a vision system processor;
providing the acquired 3D image to the vision system processor; and
finding, with the processor, a pose that simultaneously matches 3D edges in the 3D model to 3D edges in the 3D image and 3D normals in the 3D model to 3D normals in the 3D image.

2. The method as set forth in claim 1 , wherein the step of finding comprises applying weightings to 3D edges in the 3D image and 3D normals in the 3D image so as to weight use of 3D edges versus 3D normals in the image.

3. The method as set forth in claim 2 , wherein the the step of finding determines whether (a) a plane of the object provides information about alignment in directions parallel to the 3D normals, and (b) edges of the object provide information about alignment in one or more directions perpendicular to the edges, respectively.

4. The method as set forth in claim 1 , further comprising, matching the 3D edges in the 3D model to 3D edges in the 3D image using a point-to-line metric.

5. The method as set forth in claim 4 , further comprising, matching the 3D normals in the 3D model to the 3D normals in the 3D image using a point-to-plane metric.

6. The method as set forth in claim 1 , further comprising, matching the 3D normals in the 3D model to the 3D normals in the 3D image using a point-to-plane metric.

7. The method as set forth in claim 3 , wherein the step of finding includes defining a normal information matrix that represents the directions in which sufficient quantity of the information is present.

8. The method as set forth in claim 7 , further comprising, performing a principal component analysis (PCA) on the matrix to identify the information and determine availability thereof for use in the step of finding.

9. The method as set forth in claim 8 , wherein the step of performing includes evaluating the edges, respectively, for a quantity of the information contributed in respective directions and the information that is available.

10. The method as set forth in claim 9 , wherein the step of performing evaluates edges according to the following:
(a) if one of the resepective edges contribute a significant quantity of the information in a direction that is significant, then that one of the edges is assigned a high weight in the computation, and
(b) if one of the respective edges does not contribute a significant quantity of the information in a direction that is significant, or if the direction is not significant, then that one of the edges is assigned a relatively low weight in the computation.

11. The method as set forth in claim 6 , further comprising operating a linear minimization function that simultaneously minimizes a sum of distances computed using the point-to-plane metric plus a sum of distances computed using the point-to-edge metric.

12. A system for finding the pose of a 3D model in a 3D image of an object acquired by a 3D camera assembly, comprising:
a vision system processor that receives the 3D model and the acquired 3D image; and
a pose finding process that simultaneously matches 3D edges in the 3D model to 3D edges in the 3D image and 3D normals in the 3D model to 3D normals in the 3D image.

13. The system as set forth in claim 12 , wherein the pose finding process applies weightings to 3D edges in the 3D image and 3D normals in the 3D image so as to weight use of 3D edges versus 3D normals in the image.

14. The system as set forth in claim 13 , wherein the pose finding process determines whether (a) a plane of the object provides information about alignment in directions parallel to the 3D normals, and (b) edges of the object provide information about alignment in one or more directions perpendicular to the edges, respectively.

15. The system as set forth in claim 14 , wherein the pose finding process matches at least one of (a) the 3D edges in the 3D model to 3D edges in the 3D image using a point-to-line metric, and (b) the 3D normals in the 3D model to the 3D normals in the 3D image using a point-to-plane metric.

16. The system as set forth in claim 15 , wherein the pose finding process defines a normal information matrix that represents the directions in which sufficient quantity of the information is present.

17. The system as set forth in claim 16 , wherein the pose finding process performs a principal component analysis (PCA) on the matrix to identify the information, and determine availability thereof, for pose finiding.

18. The system as set forth in claim 17 , wherein the pose finding process comprises an evaluation process that evaluates the edges, respectively, for a quantity of the information contributed in respective directions and the information that is available.

19. The system as set forth in claim 18 , wherein the evaluation process evaluates pose edges according to the following:
(a) if one of the resepective edges contribute a significant quantity of the information in a direction that is significant, then that one of the edges is assigned a high weight in the computation, and
(b) if one of the respective edges does not contribute a significant quantity of the information in a direction that is significant, or if the direction is not significant, then that one of the edges is assigned a relatively low weight in the computation.

20. The system as set forth in claim 15 , further comprising a linear minimization process that simultaneously minimizes a sum of distances computed using the point-to-plane metric plus a sum of distances computed using the point-to-edge metric.

Assignees

• Cognex Corp

Inventors

• Hoelscher Andrew
• Barker Simon
• Wagman Adam
• Michael David J

Classifications

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V30/2504

  Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches · CPC title