Surface defect detection

Patent metadata

Field	Value
Publication number	US-10467502-B2
Application number	US-201815925010-A
Country	US
Kind code	B2
Filing date	Mar 19, 2018
Priority date	Mar 20, 2017
Publication date	Nov 5, 2019
Grant date	Nov 5, 2019

Abstract

A method is provided of forming a neural network for detecting surface defects in aircraft engine components. The method includes: providing (i) a pre-trained deep learning network and (ii) a learning machine network; providing a set of pixelated training images of aircraft engine components exhibiting examples of different classes of surface defect; training the trainable weights of the learning machine network on the set of training images.

First claim

The invention claimed is:

1. A method of forming a neural network for detecting surface defects in aircraft engine components, the method including:
providing (i) a pre-trained deep learning network having, in sequence, an input layer for receiving pixel values of pixelated images of objects, a set of convolutional layers, a set of fully-connected layers, and an output layer, wherein the pre-trained deep learning network has fixed weights and is pre-trained, and (ii) a learning machine network having, in sequence, an input layer, at least one hidden layer, and an output layer, wherein the values of the nodes of the input layer of the learning machine network derive from the values of the nodes of one of the convolutional layers of the pre-trained deep learning network, the weights from at least one of the layers of the learning machine network to the next layer of the learning machine network are trainable but the weights from at least another one of the layers of the learning machine network to the next layer of the learning machine network are fixed, and the nodes of the output layer of the learning machine network indicate whether a given surface defect class is displayed by a given image;
providing a set of pixelated training images of aircraft engine components exhibiting examples of different classes of surface defect, the training images being labelled with the surface defect classes exhibited by their respective components such that each class of surface defect is represented by a respective subset of the training images, and each training image being divided into one or more patches which are respective sub-areas of that image; and
training the trainable weights of the learning machine network on the set of training images by inputting each patch into the input layer of the pre-trained deep learning network and adjusting the trainable weights on the basis of a comparison between the node values of the output layer of the learning machine network and the surface defect class label of the parent training image of that patch;
whereby the layers of the pre-trained deep learning network from its input layer to its convolutional layer from which the values of the nodes of the input layer of the learning machine network derive, together with the layers of the learning machine network after the training of its trainable weights form a combined network for detecting surface defects in aircraft engine components having an input layer which is the input layer of the pre-trained deep learning network and an output layer which is the output layer of the learning machine network.

2. A method of forming a neural network according to claim 1 , wherein the pre-trained deep learning network is pre-trained such that its output layer can classify the objects into different object classes and identify the locations of the objects within the images.

3. A method of forming a neural network according to claim 1 , wherein the nodes of the output layer of the learning machine network indicate whether a given surface defect class is displayed by a given patch by specifying respective confidence levels for the presences of the different classes of surface defect within the given patch.

4. A method of forming a neural network according to claim 1 , wherein a given training image is labelled with a given surface defect class if the given training image, or any of the other training images showing the same component, displays a defect of the given class.

5. A method of forming a neural network according to claim 1 , wherein the layers of the pre-trained deep learning network counting from its input layer up to and including its convolutional layer from which the values of the nodes of the input layer of the learning machine network derive include just two, three or four convolutional layers.

6. A method of forming a neural network according to claim 1 , wherein the layers of the pre-trained deep learning network counting from its input layer up to and including its convolutional layer from which the values of the nodes of the input layer of the learning machine network derive include at least one max pooling layer.

7. A method of forming a neural network according to claim 1 , wherein the node values of the output layer of the learning machine network are thresholded to compensate for variation in image brightness across the training images, the trainable weights of the learning machine network being adjusted on the basis of a comparison between the thresholded node values of the output layer of the learning machine network and the surface defect class label of the parent training image of that patch.

8. A method of forming a neural network according to claim 1 , wherein the fixed weights of the learning machine network are set randomly.

9. A method of forming a neural network according to claim 1 , wherein the learning machine network has only one hidden layer.

10. A method of detecting surface defects in aircraft engine components, the method including:
providing the combined network formed by:
providing (i) a pre-trained deep learning network having, in sequence, an input layer for receiving pixel values of pixelated images of objects, a set of convolutional layers, a set of fully-connected layers, and an output layer, wherein the pre-trained deep learning network has fixed weights and is pre-trained, and (ii) a learning machine network having, in sequence, an input layer, at least one hidden layer, and an output layer, wherein the values of the nodes of the input layer of the learning machine network derive from the values of the nodes of one of the convolutional layers of the pre-trained deep learning network, the weights from at least one of the layers of the learning machine network to the next layer of the learning machine network are trainable but the weights from at least another one of the layers of the learning machine network to the next layer of the learning machine network are fixed, and the nodes of the output layer of the learning machine network indicate whether a given surface defect class is displayed by a given image;
providing a set of pixelated training images of aircraft engine components exhibiting examples of different classes of surface defect, the training images being labelled with the surface defect classes exhibited by their respective components such that each class of surface defect is represented by a respective subset of the training images, and each training image being divided into one or more patches which are respective sub-areas of that image; and
training the trainable weights of the learning machine network on the set of training images by inputting each patch into the input layer of the pre-trained deep learning network and adjusting the trainable weights on the basis of a comparison between the node values of the output layer of the learning machine network and the surface defect class label of the parent training image of that patch;
whereby the layers of the pre-trained deep learning network from its input layer to its convolutional layer from which the values of the nodes of the input layer of the learning machine network derive, together with the layers of the learning machine network after the training of its trainable weights form a combined network for detecting surface defects in aircraft engine components having an input layer which is the input layer of the pre-trained deep learning network and an output layer which is the output layer of the learning machine network;
providing a pixelated actual image of an aircraft engine component, the actual image being divided into one or more patches which are respective sub-areas of that image; and
inputting each patch into the input layer of the combined ne

Assignees

• Rolls Royce Plc

Inventors

• Ren Ruo Xu
• Hung Terence
• Tan Kay Chen

Classifications

• G06T2207/30164

  Workpiece; Machine component · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T7/0004 Primary

  Industrial image inspection · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title