Vision system and analytical method for planar surface segmentation

Patent metadata

Field	Value
Publication number	US-10115035-B2
Application number	US-201514592133-A
Country	US
Kind code	B2
Filing date	Jan 8, 2015
Priority date	Jan 8, 2015
Publication date	Oct 30, 2018
Grant date	Oct 30, 2018

Abstract

A vision system is configured to dynamically inspect an object in a field of view. This includes capturing, using a camera, three-dimensional (3D) point cloud data of the field of view and transforming each of the points of the 3D point cloud data into a plurality of tangential surface vectors. Surface normal vectors are determined for each of the points of the 3D point cloud data based upon the plurality of tangential surface vectors. Distribution peaks in the surface normal vectors are detected employing a unit sphere mesh. Parallel planes are separated using the distance distribution peaks. A radially bounded nearest neighbor strategy combined with a process of nearest neighbor searching based upon cell division is executed to segment a planar patch. A planar surface is identified based upon the segmented planar patch.

First claim

The invention claimed is:

1. A method for employing a vision system to dynamically inspect an object in a field of view, comprising:
capturing three-dimensional (3D) point cloud data of the field of view;
transforming each point of the 3D point cloud data into a plurality of tangential surface vectors;
determining surface normal vectors for each point of the 3D point cloud data based upon the plurality of tangential surface vectors;
detecting distribution peaks in the surface normal vectors employing a unit sphere mesh, including inscribing an icosahedron into the unit sphere mesh and recursively subdividing each triangle therein to generate triangulations having a multiplicity of triangle facets and mesh vertices and detecting the distribution peaks based thereon;
separating parallel planes using the detected distribution peaks, wherein separating the parallel planes includes executing, using a controller, a radially bounded nearest neighbor strategy combined with a process of nearest neighbor searching based upon cell division to segment the separated parallel planes in the same orientation into a plurality of planar patches; and
identifying a planar surface of the object based upon the segmented planar patches.

2. The method of claim 1 , wherein transforming the 3D point cloud data into a plurality of tangential surface vectors comprises taking a differential of two neighboring points to transform each of the points in the 3D point cloud data into a tangential surface vector.

3. The method of claim 1 , wherein determining the surface normal vectors for each of the points of the 3D point cloud data based upon the plurality of tangential surface vectors includes:
identifying a neighborhood of points in a surface area around each of the points in the 3D point cloud;
determining a tangential surface vector for each point in the neighborhood of points; and
calculating a cross-product of the tangential surface vectors for the neighborhood of points.

4. The method of claim 3 , wherein identifying a neighborhood of points in a surface area around each of the points in the 3D point cloud comprises identifying a surface area of 7×7 points for each of the points in the 3D point cloud data.

5. The method of claim 1 , wherein determining surface normal vectors for the plurality of tangential surface vectors comprises:
identifying a neighborhood of points around each of the points in the 3D point cloud data;
determining tangential surface vectors associated with the neighborhood of points; and
calculating a cross-product of the tangential surface vectors to determine a surface normal vector for each of the points in the 3D point cloud data.

6. The method of claim 1 , wherein detecting distribution peaks in the surface normal vectors employing a unit sphere mesh comprises applying a discrete Laplacian-Beltrami (LB) operator on the mesh vertices of the triangles associated with the unit sphere mesh to detect the distribution peaks, including:
identifying reference points on the unit sphere mesh,
applying a triangular mesh mask to the unit sphere mesh, wherein the triangular mesh mask approximates a second derivative, and
determining peak points on the unit sphere mesh based upon the applying of the triangular mesh mask to unit sphere mesh.

7. The method of claim 1 , wherein capturing the 3D point cloud data of the field of view comprises capturing the 3D point cloud data of the field of view using a digital camera.

8. The method of claim 1 , wherein separating the parallel planes using the distribution peaks comprises segmenting the parallel planes using the distance distribution along a planar normal direction.

9. A method for dynamically processing a three-dimensional (3D) point cloud associated with a field of view, comprising:
transforming each point of the 3D point cloud into a plurality of tangential surface vectors;
determining surface normal vectors for each point of the 3D point cloud based upon the plurality of tangential surface vectors;
detecting distribution peaks in the surface normal vectors employing a unit sphere mesh, including inscribing an icosahedron into the unit sphere mesh and recursively subdividing each triangle therein to generate triangulations having a multiplicity of triangle facets and mesh vertices and detecting the distribution peaks based thereon;
separating parallel planes using the detected distribution peaks wherein separating the parallel planes includes executing, using a controller, a radially bounded nearest neighbor strategy combined with a process of nearest neighbor searching based upon cell division to segment the separated parallel planes in the same orientation into a plurality of planar patches; and
identifying a planar surface based upon the segmented planar patches.

10. The method of claim 9 , wherein transforming the 3D point cloud data into a plurality of tangential surface vectors comprises taking a differential of two neighboring points to transform each of the points in the 3D point cloud data into a tangential surface vector.

11. The method of claim 9 , wherein determining the surface normal vectors for each of the points of the 3D point cloud data based upon the plurality of tangential surface vectors includes:
identifying a neighborhood of points in a surface area around each of the points in the 3D point cloud;
determining a tangential surface vector for each point in the neighborhood of points; and
calculating a cross-product of the tangential surface vectors for the neighborhood of points.

12. The method of claim 11 , wherein identifying a neighborhood of points in a surface area around each of the points in the 3D point cloud comprises identifying a surface area of 7×7 points for each of the points in the 3D point cloud data.

13. The method of claim 9 , wherein determining surface normal vectors for the plurality of tangential surface vectors comprises:
identifying a neighborhood of points around each of the points in the 3D point cloud data;
determining tangential surface vectors associated with the neighborhood of points; and
calculating a cross-product of the tangential surface vectors to determine a surface normal vector for each of the points in the 3D point cloud data.

14. The method of claim 9 , wherein detecting distribution peaks in the surface normal vectors employing a unit sphere mesh comprises applying a discrete Laplacian-Beltrami (LB) operator on the mesh vertices of the triangles associated with the unit sphere mesh to detect the distribution peaks, including:
identifying reference points on the unit sphere mesh,
applying a triangular mesh mask to the unit sphere mesh, wherein the triangular mesh mask approximates a second derivative, and
determining peak points on the unit sphere mesh based upon the applying of the triangular mesh mask to unit sphere mesh.

15. The method of claim 9 , wherein capturing the 3D point cloud data of the field of view comprises capturing the 3D point cloud data of the field of view using a digital camera.

16. The method of claim 9 , wherein separating parallel planes using the distribution peaks comprises separating parallel planes using the distance distribution along a planar normal direction.

17. A vision system for dynamically inspecting an object in a field of view, comprising:
a digital camera signally connected to a camera controller signally connected to an analytical controller;
the digital camera configured to capture a bitmap image file including a three-dimensional (3D) image of the field of view;
the camera controller executing digi

Assignees

• Gm Global Tech Operations Llc
• Sungkyunkwan Univ Foundation For Corporate Collaboration
• Sungkyunkwan Univ Foundation For Corporation Collaboration

Inventors

• Lee Sukhan
• Nguyen Hung Huu
• Kim Jaewoong
• Shi Jianying

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T7/0004

  Industrial image inspection · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06K9/00201

  Physics · mapped topic

• G06K9/6209

  Physics · mapped topic