Visual anomaly detection system

Patent metadata

Field	Value
Publication number	US-10475174-B2
Application number	US-201715480670-A
Country	US
Kind code	B2
Filing date	Apr 6, 2017
Priority date	Apr 6, 2017
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

A generative adversarial network (GAN) system includes a generator neural sub-network configured to receive one or more images depicting one or more objects. The generator neural sub-network also is configured to generate a foreground image and a background image based on the one or more images that are received, the generator neural sub-network configured to combine the foreground image with the background image to form a consolidated image. The GAN system also includes a discriminator neural sub-network configured to examine the consolidated image and determine whether the consolidated image depicts at least one of the objects. The generator neural sub-network is configured to one or more of provide the consolidated image or generate an additional image as a training image used to train another neural network to automatically identify the one or more objects in one or more other images.

First claim

What is claimed is:

1. A generative adversarial network (GAN) system comprising:
a generator neural sub-network configured to receive one or more images depicting one or more objects, the generator neural sub-network also configured to generate a foreground image and a background image based on the one or more images that are received, the generator neural sub-network configured to combine the foreground image with the background image to form a consolidated image;
a discriminator neural sub-network configured to examine the consolidated image and determine whether the consolidated image depicts at least one of the objects; and
a neural network configured to receive the consolidated image from the generator neural sub-network as a training image, the neural network configured to be trained to automatically identify the one or more objects in one or more other images,
wherein the neural network identifies at least one damaged coating on the one or more objects after being trained using the consolidated image,
wherein the one or more objects further comprise at least one component of a gas turbine engine onto which the at least one damaged coating is disposed.

2. The GAN system of claim 1 , wherein the generator neural sub-network is configured to generate the foreground image as a generated image of at least one of the objects without a background from the one or more images received by the generator neural sub-network, and
wherein the generator neural sub-network identifies at least one darker quasi-circular shape on a lighter background as spalling of a thermal barrier coating.

3. The GAN system of claim 1 , wherein the generator neural sub-network is configured to generate the background image as a generated image of a background from the one or more images received by the generator neural sub-network without the one or more objects being included in the background image, and
wherein the generator neural sub-network forms the consolidated image by overlaying the foreground image onto the background image.

4. The GAN system of claim 1 , wherein the generator neural sub-network is configured to generate one or more of the foreground image or the background image based on one or more distributions of pixel characteristics from the one or more images received by the generator neural sub-network, and
wherein the generator neural sub-network forms the consolidated image by overlaying the background image onto the foreground image.

5. The GAN system of claim 1 , wherein the generator neural sub-network is configured to receive the one or more images without pixels of the one or more images that are received being labeled as depicting one or more object classes, and
wherein the generator neural sub-network forms the consolidated image by inserting one or more pixels of the foreground image into the background image in corresponding locations of the one or more pixels in the foreground image.

6. The GAN system of claim 1 , wherein the generator neural sub-network is configured to generate the consolidated image as the training image with pixels of the training image associated with one or more object classes, and
wherein the generator neural sub-network calculates at least one score for at least one category of the one or more objects using at least one linear classification.

7. A method comprising:
receiving one or more images depicting one or more objects at a generator neural sub-network of a generative adversarial network (GAN) system;
generating a foreground image and a background image using the generator neural sub-network and based on the one or more images that are received;
combining the foreground image with the background image to form a consolidated image using the generator neural sub-network;
examining the consolidated image and determine whether the consolidated image depicts at least one of the objects using a discriminator neural sub-network of the GAN system,
wherein one or more of the consolidated image or an additional generated image is configured to be provided to another artificial neural network as a training image for training the artificial neural network to automatically identify the one or more objects in one or more other images, and
wherein the artificial neural network automatically identifies the one or more objects by determining that the foreground image is at least one of darker than, lighter than, and a different color than the background image, the one or more objects include spalling of a thermal barrier coating; and
identifying, at the generator neural sub-network, at least one darker quasi-circular shape on a lighter background as the spalling of a thermal barrier coating.

8. A generative adversarial network (GAN) system comprising:
a generator neural sub-network configured to receive one or more images depicting one or more objects, the generator neural sub-network also configured to generate a foreground image and a background image based on the one or more images that are received, the generator neural sub-network configured to combine the foreground image with the background image to form a consolidated image;
a discriminator neural sub-network configured to examine the consolidated image and determine whether the consolidated image depicts at least one of the objects; and
a neural network configured to receive the consolidated image as a training image, the neural network configured to be trained to automatically identify the one or more objects in one or more other images using the training image,
wherein the generator neural sub-network is configured to receive the one or more images without pixels of the one or more images that are received being labeled as depicting one or more object classes, and
wherein the one or more objects further comprise at least one component of a gas turbine engine onto which the at least one damaged coating is disposed.

9. The GAN system of claim 8 , wherein the one or more objects include damage to the at least one component,
wherein the at least one component comprises a turbine blade.

10. The GAN system of claim 8 , wherein the generator neural sub-network is configured to generate the foreground image as a generated image of at least one of the objects without a background from the one or more images received by the generator neural sub-network, and
wherein the generator neural sub-network forms the consolidated image by inserting one or more pixels of the background image into the foreground image in corresponding locations of the one or more pixels in the background image.

11. The GAN system of claim 8 , further comprising at least one field programmable gate array, wherein the generator neural sub-network is configured to generate the background image as a generated image of a background from the one or more images received by the generator neural sub-network without the one or more objects being included in the background image.

12. The GAN system of claim 8 , wherein the generator neural sub-network is configured to generate one or more of the foreground image or the background image based on one or more distributions of pixel characteristics from the one or more images received by the generator neural sub-network, and
wherein less than 50% of the pixels in the training images are labeled.

13. The GAN system of claim 8 , wherein the generator neural sub-network is configured to generate the consolidated image as the training image with pixels of the training image associated with one or more object classes, and
wherein less than 10% of the pixels in the training images are labeled.

Assignees

• Gen Electric

Inventors

• Lim Ser Nam
• Diwinsky David
• Lin Yen-Liang
• Bian Xiao

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T2207/30164

  Workpiece; Machine component · CPC title

• G06T7/0004 Primary

  Industrial image inspection · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title