Multi-view 3D object recognition from a point cloud and change detection

Patent metadata

Field	Value
Publication number	US-9619691-B2
Application number	US-201514641171-A
Country	US
Kind code	B2
Filing date	Mar 6, 2015
Priority date	Mar 7, 2014
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

A method of detecting objects in three-dimensional (3D) point clouds and detecting differences between 3D point clouds and the objects therein is disclosed. A method includes receiving a first scene 3D point cloud and a second scene 3D point cloud, wherein the first scene 3D point cloud and the second scene 3D point cloud include first and second target objects, respectively; aligning the first scene 3D point cloud and the second scene 3D point cloud; detecting the first and second target objects from the first scene 3D point cloud and the second scene 3D point cloud, respectively; comparing the detected first target object with the detected second target object; and identifying, based on the comparison, one or more differences between the detected first target object and the detected second target object. Further aspects relate to detecting changes of target objects within scenes of multiple 3D point clouds.

First claim

What is claimed is:

1. A computer-implemented method of recognizing an object in a three dimensional point cloud, the method comprising:
receiving a first scene three-dimensional (3D) point cloud and a first target object 3D point cloud;
projecting the first scene 3D point cloud into a first plurality of two-dimensional (2D) depth images;
projecting the first target object 3D point cloud into a second plurality of 2D depth images;
detecting the second plurality of 2D depth images in the first plurality of 2D depth images, resulting in a first plurality of 2D detection locations;
re-projecting, into 3D space, the first plurality of 2D detection locations; and
determining 3D locations of the detected first target object from the re-projected first plurality of 2D detection locations to detect the first target object.

2. The computer-implemented method of recognizing an object in a 3D point cloud of claim 1 , further comprising:
receiving a second scene three-dimensional (3D) point cloud and a second target object 3D point cloud;
projecting the second scene 3D point cloud into a third plurality of two-dimensional (2D) depth images;
projecting the second target object 3D point cloud into a fourth plurality of 2D depth images;
detecting the fourth plurality of 2D depth images in the third plurality of 2D depth images, resulting in a second plurality of 2D detection locations;
re-projecting, into 3D space, the second plurality of 2D detection locations; and
determining 3D locations of the detected second target object from the re-projected second plurality of 2D detection locations to detect the second target object.

3. The computer-implemented method of recognizing an object in a 3D point cloud of claim 2 , further comprising:
aligning the first scene 3D point cloud and the second scene 3D point cloud;
comparing the detected first target object with the detected second target object; and
identifying, based on the comparison, one or more differences between the detected first target object and the detected second target object.

4. The computer-implemented method of recognizing an object in a 3D point cloud of claim 1 , wherein detecting the first target object is performed using a gradient-based detection algorithm.

5. The computer-implemented method of recognizing an object in a 3D point cloud of claim 1 , wherein detecting the first target object is performed using a template-matching detection algorithm.

6. The computer-implemented method of recognizing an object in a 3D point cloud of claim 1 , wherein detecting the first target object is performed using a BRIEF-based detection algorithm.

7. The computer-implemented method of recognizing an object in a 3D point cloud of claim 1 , wherein projecting the first plurality of images and the second plurality of images comprises transforming the first scene 3D point cloud and the second scene 3D point cloud into a plurality of 2D images from multiple viewing angles.

8. The computer-implemented method of recognizing an object in a 3D point cloud of claim 7 , wherein transforming the first scene 3D point cloud and the second scene 3D point cloud comprises using depth information.

9. The method for recognizing an object in a 3D point cloud of claim 1 , wherein detecting the first target object further comprises:
determining a pixel value for each pixel of the 2D depth image;
comparing the pixel value of at least two neighboring pixels in x- and y-directions; and
computing a gradient based on the comparison.

10. A computer-implemented method of detecting differences between three-dimensional (3D) point clouds, the method comprising:
receiving a first scene 3D point cloud and a second scene 3D point cloud, wherein the first scene 3D point cloud and the second scene 3D point cloud include first and second target objects, respectively;
aligning the first scene 3D point cloud and the second scene 3D point cloud;
detecting the first and second target objects from the first scene 3D point cloud and the second scene 3D point cloud, respectively;
comparing the detected first target object with the detected second target object;
and identifying, based on the comparison, one or more differences between the detected first target object and the detected second target object;
wherein detecting the first target object further comprises:
receiving a first scene 3D point cloud and a first target object 3D point cloud;
projecting the scene 3D point cloud into a first plurality of two-dimensional (2D) depth images;
projecting the first target object 3D point cloud into a second plurality of 2D depth images;
detecting the second plurality of 2D depth images in the first plurality of 2D depth images, resulting in a first plurality of 2D detection locations;
re-projecting, into 3D space, the first plurality of 2D detection locations; and
determining 3D locations of the detected first target object from the re-projected first plurality of 2D detection locations to detect the first target object.

11. The computer-implemented method of detecting differences between 3D point clouds of claim 10 , wherein detecting the first target object is performed using a gradient-based detection algorithm.

12. The computer-implemented method of detecting differences between 3D point clouds of claim 10 , wherein detecting the first target object is performed using a template-matching detection algorithm.

13. The computer-implemented method of detecting differences between 3D point clouds of claim 10 , wherein detecting the first target object is performed using a BRIEF-based detection algorithm.

14. The computer-implemented method of detecting differences between 3D point clouds of claim 10 , wherein:
projecting the first plurality of images and the second plurality of images comprises transforming the first scene 3D point cloud and the second scene 3D point cloud into a plurality of 2D images from multiple viewing angles; and
transforming the first scene 3D point cloud and the second scene 3D point cloud comprises using depth information.

15. The computer-implemented method of detecting differences between 3D point clouds of claim 10 , wherein detecting the first target object further comprises:
determining a pixel value for each pixel of the 2D depth image;
comparing the pixel value of at least two neighboring pixels in x- and y-directions; and
computing a gradient based on the comparison.

16. A system for detecting changes and recognizing target objects in a three-dimensional (3D) point cloud, comprising:
a computing device including a processor and a memory communicatively coupled to the processor, the memory storing computer-executable instructions which, when executed by the processor, cause the system to perform a method comprising:
receiving a first scene 3D point cloud representing an industrial scene at a first time and a second scene 3D point cloud representing the industrial scene at a second time after the first time, wherein the first scene 3D point cloud and the second scene 3D point cloud include first and second target objects, respectively;
aligning the first scene 3D point cloud and the second scene 3D point cloud;
detecting the first and second target objects from the first scene 3D point cloud and the second scene 3D point cloud, respectively;
wherein detecting the first target object further comprises:
receiving a first scene 3D point cloud and a first target object 3D point cloud;
projecting the first scene 3D point cloud into a first plurality of two-dimensional (2D) depth images;
projecting the first target object 3D

Assignees

• Pang Guan
• Huang Jing
• Anvar Amir
• Casey Michael Brandon
• Fisher Christopher Lee
• You Suya
• Neumann Ulrich
• Univ Southern California
• Chevron Usa Inc

Inventors

• Pang Guan
• Huang Jing
• Anvar Amir
• Casey Michael Brandon
• Fisher Christopher Lee
• You Suya
• Neumann Ulrich

Classifications

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• H04N13/0011

  Electricity · mapped topic

• G06T7/0044

  Physics · mapped topic

• G06K2209/19

  Physics · mapped topic

• G06T2207/30108

  Industrial image inspection · CPC title