Structure defect detection using machine learning algorithms

Patent metadata

Field	Value
Publication number	US-2020175352-A1
Application number	US-201916570414-A
Country	US
Kind code	A1
Filing date	Sep 13, 2019
Priority date	Mar 14, 2017
Publication date	Jun 4, 2020
Grant date	—

Abstract

Structure defect detection is performed using computer-implemented arrangements employing machine learning algorithms in the form of neural networks. In one arrangement, a convolutional neural network is trained using a database of images formed to optimize accuracy of the convolutional neural network to detect, for example, a crack in a concrete surface. A two-stage scanning process each performing a plurality of scans of a test image is incorporated in the foregoing arrangement of convolutional neural network, with the two-stages forming overlapping capture areas to reduce likelihood of a crack lying on a boundary of the individual scans going undetected. Also, region-based convolutional neural networks are trained to detect various types of defects.

First claim

1 . A computer-implemented method of analyzing an image of a surface to detect a defect in the surface, comprising:
receiving the image of the surface; processing the image using a machine learning algorithm configured to detect the defect, the machine learning algorithm comprising a convolutional neural network including:
at least one convolution layer; and
at least one max pooling layer;
and displaying the image with location of the defect being indicated if determined to be present by the convolutional neural network.

2 . The computer-implemented method of claim 1 wherein said processing includes:
scanning a first set of non-overlapping areas of the image;
scanning a second set of non-overlapping areas of the image each which overlap more than one of the first set of non-overlapping areas so as to capture information at edges of the first set of non-overlapping areas which is otherwise unavailable to the convolutional neural network from the scanning of the first set of non-overlapping areas.

3 . The computer-implemented method of claim 1 wherein the convolutional neural network comprises:
an input layer having a height of n pixels, a width of n pixels, and a depth of d channels;
said at least one convolution layer comprising a preliminary convolution layer, a secondary convolution layer, a tertiary convolution layer and a quaternary convolution layer;
said at least one pooling layer comprising a preliminary pooling layer and a secondary pooling layer;
the preliminary convolution layer having a height of Hc 1 pixels, a width of Hc 1 pixels, and a depth of Dc 1 channels formed by a first convolution operator having a height of hc 1 pixels, a width of hc 1 pixels, and a depth of dc 1 channels with a stride of sc 1 performed upon the input layer;
wherein Hc 1 =[( n−hc 1 )/ sc 1 ]+1;
wherein Dc 1 =dc 1 ;

the preliminary pooling layer having a height of Hp 1 pixels, a width of Hp 1 pixels, and a depth of Dp 1 channels formed by a first pooling operator having a height of hp 1 pixels and a width of hp 1 pixels with a stride of sp 1 performed on the preliminary convolution layer;
wherein Hp 1 =[( Hc 1 −hp 1 )/ sp 1 ]+1;
wherein Dp 1 =Dc 1 ;

the secondary convolution layer having a height of Hc 2 pixels, a width of Hc 2 pixels, and a depth of Dc 2 channels formed by a second convolution operator having a height of hc 2 pixels, a width of hc 2 pixels, and a depth of dc 2 channels with a stride of sc 2 performed upon the preliminary pooling layer;
wherein Hc 2 =[( Hp 1 −hc 2 )/ sc 2 ]+1;
wherein Dc 2 =dc 2

the secondary pooling layer having a height of Hp 2 pixels, a width of Hp 2 pixels, and a depth of Dp 2 channels formed by a second pooling operator having a height of hp 2 pixels and a width of hp 2 pixels with a stride of sp 2 performed upon the secondary convolution layer;
wherein Hp 2 =[( Hc 1 −hp 2 )/ sp 2 ]+1;
wherein Dp 2 =Dc 2

the tertiary convolution layer having a height of Hc 3 pixels, a width of Hc 3 pixels, and a depth of Dc 3 channels formed by a third convolution operator having a height of hc 3 pixels, a width of hc 3 pixels, and a depth of dc 3 channels with a stride of sc 3 that is performed upon the secondary pooling layer;
wherein Hc 3 =[( Hp 2 −hc 3 )/ sc 3 ]+1;
wherein Dc 3 =dc 3 ;

an activation layer having a height of Ha 1 pixels, a width of Ha 1 pixel, and a depth of Da 1 channels formed by a nonlinear activation function operator performed upon the tertiary convolution layer;
wherein Ha 1 =Hc 3 ;
wherein Da 1 =Dc 3 ;

the quaternary convolution layer having a height of Hc 4 pixels, a width of Hc 4 pixels, and a depth of Dc 4 channels formed by a fourth convolution operator having a height of hc 4 pixel, a width of hc 4 pixel, and a depth of dc 4 channels with a stride of sc 4 performed upon the activation layer;
wherein Hc 4 =[( Ha 1 −hc 4 )/ sc 4 ]+1;
wherein Dc 4 =dc 4 ;

and a softmax layer having a height of Sm 1 pixels, a width of Sm 1 pixels, and a depth of Dsm 1 channels formed by a softmax operator performed upon the quaternary convolution layer such that a continuously extending line in an image can be detected;
wherein Sm 1 =Hc 4 ;
wherein Dsm 1 =Dc 4 .

4 . The computer-implemented method of claim 3 wherein the first convolution operator has a height of 20 pixels, a width of 20 pixels, and a depth of 3 channels with a stride of 2.

5 . The computer-implemented method of claim 3 wherein the first pooling operator has a height of 7 pixels and a width of 7 pixels with a stride of 2.

6 . The computer-implemented method of claim 3 wherein the second convolution operator has a height of 15 pixels, a width of 15 pixels, and a depth of 24 channels with a stride of 2.

7 . The computer-implemented method of claim 3 wherein the second pooling operator has a height of 4 pixels and a width of 4 pixels with a stride of 2.

8 . The computer-implemented method of claim 3 wherein the third convolution operator has a height of 10 pixels, a width of 10 pixels, and a depth of 48 channels with a stride of 2.

9 . The computer-implemented method of claim 3 wherein the fourth convolution operator has height of 1 pixel, a width of 1 pixel, and a depth of 96 channels with a stride of 1.

10 . The computer-implemented method of claim 1 wherein the convolutional neural network comprises:
an input layer having a height of 256 pixels, a width of 256 pixels, and a depth of 3 channels;
said at least one convolution layer comprising a preliminary convolution layer, a secondary convolution layer, a tertiary convolution layer and a quaternary convolution layer;
said at least one pooling layer comprising a preliminary pooling layer and a secondary pooling layer;
the preliminary convolution layer having a height of 119 pixels, a width of 119 pixels, and a depth of 24 channels formed by a first convolution operator having a height of 20 pixels, a width of 20 pixels, and a depth of 3 channels with a stride of 2 performed upon the input layer;
the preliminary pooling layer having a height of 57 pixels, a width of 57 pixels, and a depth of 24 channels formed by a first pooling operator having a height of 7 pixels and a width of 7 pixels with a stride of 2 performed on the preliminary convolution layer;
the secondary convolution layer having a height of 22 pixels, a width of 22 pixels, and a depth of 48 channels formed by a second convolution operator having a height of 15 pixels, a width of 15 pixels, and a depth of 24 channels with a stride of 2 performed upon the preliminary pooling layer;
the secondary pooling layer having a height of 10 pixels, a width of 10 pixels, and a depth of 48 channels formed by a second pooling operator having a height of 4 pixels and a width of 4 pixels with a stride of 2 performed upon the secondary convolution layer;
the tertiary convolution layer having a height of 1 pixel, a width of 1 pixel, and a depth of 96 channels formed by a third convolution operator having a height of 10 pixels, a width of 10 pixels, and a depth of 48 channels with a stride of 2 performed upon the secondary pooling layer;
an activation layer having a height of 1 pixel, a width of 1 pixel, and a depth of 96 channels formed by a nonlinear activation function operator performed upon the tertiary convolution layer;
the quaternary convolution layer having a height of 1 pixel, a width of 1 pixel, and a depth of 2 channels formed by a fourth convolution operator having a height of 1 pixel, a width of 1 pixel, and a depth of 96 channels with

Assignees

• Univ Manitoba

Inventors

• Cha Young Jin
• Choi Wooram

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/30132

  Masonry; Concrete · CPC title

• G06N3/04 Primary

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title