Neural network point cloud generation system

Patent metadata

Field	Value
Publication number	US-10262243-B2
Application number	US-201715604012-A
Country	US
Kind code	B2
Filing date	May 24, 2017
Priority date	May 24, 2017
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

A system includes one or more processors and a memory that stores a generative adversarial network (GAN). The one or more processors are configured to receive a low resolution point cloud comprising a set of three-dimensional (3D) data points that represents an object. A generator of the GAN is configured to generate a first set of generated data points based at least in part on one or more characteristics of the data points in the low resolution point cloud, and to interpolate the generated data points into the low resolution point cloud to produce a super-resolved point cloud that represents the object and has a greater resolution than the low resolution point cloud. The one or more processors are further configured to analyze the super-resolved point cloud for detecting one or more of an identity of the object or damage to the object.

First claim

What is claimed is:

1. A system comprising:
a memory that stores a generative adversarial network (GAN);
one or more processors configured to receive a low resolution point cloud comprising a set of three-dimensional (3D) data points, the low resolution point cloud representing an object, the one or more processors configured to input the low resolution point cloud to the GAN for a generator of the GAN to generate a first set of generated data points based at least in part on one or more characteristics of the data points in the low resolution point cloud, the generator further configured to interpolate the generated data points into the low resolution point cloud to produce a super-resolved point cloud that represents the object and has a greater resolution than the low resolution point cloud;
wherein the one or more processors are further configured to analyze the super-resolved point cloud for detecting one or more of an identity of the object or damage to the object; and
wherein the generator of the GAN interpolates the generated data points into the low resolution point cloud at coordinate positions between adjacent data points in the low resolution point cloud.

2. The system of claim 1 , wherein the object is a turbine engine and the one or more processors are configured to analyze the super-resolved point cloud to detect damage to a coating of the turbine engine.

3. The system of claim 1 , wherein a discriminator of the GAN is configured to receive the super-resolved point cloud from the generator and to predict whether the object in the super-resolved point cloud is one or more of similar or identical to an object represented in one or more high resolution training point clouds.

4. The system of claim 3 , wherein, responsive to the discriminator predicting that the object in the super-resolved point cloud is not similar or identical to any object represented in any of the one or more high resolution training point clouds, the generator of the GAN is configured to generate a different, second set of generated data points that is interpolated into the low resolution point cloud to produce a revised super-resolved point cloud.

5. The system of claim 4 , wherein the generator of the GAN on the memory is an artificial neural network having artificial neurons that apply weighted functions to the one or more characteristics of the data points in the low resolution point cloud to generate the generated data points, and, responsive to the discriminator of the GAN predicting that the object in the super-resolved point cloud is not similar or identical to any object represented in any of the one or more high resolution training point clouds, the generator is configured to change one or more weights applied by the artificial neurons in the weighted functions prior to generating the second set of generated data points.

6. The system of claim 1 , wherein the one or more processors are configured to receive the low resolution point cloud from a time-of-flight range imaging camera.

7. The system of claim 1 , wherein the generator of the GAN is configured to generate an amount of the generated data points for the super-resolved point cloud that is at least three times more than a total amount of the data points in the low resolution point cloud.

8. The system of claim 1 , wherein the one or more characteristics of the data points in the low resolution point cloud on which the generator of the GAN is configured to generate the generated data points includes one or more of 3D position coordinates, intensities, colors, or relative positions of the data points.

9. The system of claim 1 , wherein the generator of the GAN is configured to generate the generated data points based at least in part on a determined distribution of one or more characteristics of data points in one or more training point clouds received by the GAN from the one or more processors during a training phase of the GAN.

10. A method comprising:
obtaining a low resolution point cloud comprising a set of three-dimensional (3D) data points, the low resolution point cloud representing an object;
inputting the low resolution point cloud to a generator of a generative adversarial network (GAN) trained to generate a first set of generated data points based at least in part on one or more characteristics of the data points in the low resolution point cloud, the generator further configured to interpolate the generated data points into the low resolution point cloud to produce a super-resolved point cloud that represents the object and has a greater resolution than the low resolution point cloud; and
analyzing the super-resolved point cloud for detecting one or more of an identity of the object or damage to the object;
wherein the generator of the GAN interpolates the generated data points into the low resolution point cloud at coordinate positions between adjacent data points in the low resolution point cloud.

11. The method of claim 10 , wherein the object is a turbine engine and the super-resolved point cloud is analyzed to detect damage to a coating of the turbine engine.

12. The method of claim 10 , further comprising predicting, using a discriminator of the GAN, whether the object in the super-resolved point cloud is one or more of similar or identical to an object represented in one or more high resolution training point clouds.

13. The method of claim 12 , wherein, responsive to the discriminator predicting that the object in the super-resolved point cloud is not similar or identical to any object represented in any of the one or more high resolution training point clouds, the method includes generating, using the generator, a different, second set of generated data points that is interpolated into the low resolution point cloud to produce a revised super-resolved point cloud.

14. The method of claim 13 , wherein the generator of the GAN is an artificial neural network having artificial neurons that apply weighted functions to the one or more characteristics of the data points in the low resolution point cloud to generate the generated data points, and, responsive to the discriminator predicting that the object in the super-resolved point cloud is not similar or identical to any object represented in any of the one or more high resolution training point clouds, the method includes changing one or more weights applied by the artificial neurons in the weighted functions prior to generating the second set of generated data points.

15. The method of claim 10 , further comprising training the GAN using one or more training point clouds prior to inputting the low resolution point cloud to the generator, the generator of the GAN configured to determine a distribution of one or more characteristics of data points in the one or more training point clouds and use the distribution of the one or more characteristics to generate the first set of generated data points.

16. The method of claim 10 , wherein the generator is configured to generate an amount of the generated data points for the super-resolved point cloud that is at least three times more than a total amount of the data points in the low resolution point cloud.

17. A system comprising:
a generator of a generative adversarial network (GAN) comprising one or more processors, the generator configured to receive a low resolution point cloud representing an object, the low resolution point cloud including three-dimensional (3D) data points, the generator configured to generate a first set of generated 3D data points and to interpolate the generated data points into the low resolution point

Assignees

• Gen Electric

Inventors

• Lim Ser Nam
• Zheng Jingjing
• Luo Jiajia
• Diwinsky David Scott

Classifications

• G06T3/4053 Primary

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/30108

  Industrial image inspection · CPC title