A self-learning method of generative adversarial multi-headed attention neural network for aero-engine data reconstruction

Patent metadata

Field	Value
Publication number	US-2025036924-A1
Application number	US-202218548204-A
Country	US
Kind code	A1
Filing date	Oct 28, 2022
Priority date	Oct 24, 2022
Publication date	Jan 30, 2025
Grant date	—

Abstract

A generative adversarial multi-headed attention neural network self-learning method for aero-engine data reconstruction belongs to the field of end-to-end self-learning of aero-engine missing data. First, the samples are pre-processed, and the machine learning algorithm is used to pre-fill the normalized data first, and the pre-filled information is involved in the network training as part of the training information. Second, a generative adversarial multi-headed attention network model is constructed and the trained sample set is used to train the generative adversarial multi-headed attention network model. Finally, the samples are generated using the trained sample generator G. The method uses the generative adversarial network to better learn the distribution information of the data, and uses parallel convolution and multi-headed attention mechanism to fully exploit the spatial and temporal information among the aero-engine data.

First claim

1 . A generative adversarial multi-head attention neural network self-learning method for aero-engine data reconstruction, comprising the following steps:
step S1: preprocessing a sample 1) dividing an aero-engine data set with a missing value into a training sample set and a test sample set, wherein the training sample set is used for training a model, and the test sample set is used for checking the model after training; assuming that the aero-engine data set has n attributes, then the aero-engine data set is uniformly represented by X={X 1 , X 2 , . . . X n }; 2) marking the missing value since X contains the missing value, a missing item is represented by NAN, and an unmissing item is an original value, constructing a mask matrix M equal to X in size, marking a corresponding position of the mask matrix as 0 for the missing item in X, and marking a corresponding position of the mask matrix as 1 for the unmissing item in X, thus to mark missing data and unmissing data; 3) making different features have the same scale through standardization; for the unmissing item, using the following formula to standardize all sensor data,

X
i
′

=

X
i

-

mean
i

σ
i

⁢
i

∈

(

1
,
2
,

…
⁢

n

)

(
1
)

wherein X′ i represents standardized data of a feature i, X i represents original data of the feature i, mean i represents a mean value of the feature i, and σ i represents a variance of the feature i; for the missing item, replacing NAN with 0 to finally obtain standardized multivariate time series data X′={X′ 1 , X′ 2 , . . . X′ n };
4) constructing time series samples by a sliding window method
for X′ and M, sliding in a time dimension by a sliding window method to extract time information of the sample and construct a series of time series samples of n×Windowsize, wherein n is a feature dimension of the samples, and Windowsize is a window size, i.e., reconstructing X′ and M into the form of m×n×Windowsize, wherein m is a sample size which depends on an original sample size;
step S2: conducting pre-imputation
in order to make the data generated by a network better fit original data distribution, adopting a machine learning algorithm to pre-impute X′, and using the pre-imputed information as partial training information X pre to participate in network training;
step S3: constructing a generative adversarial multi-head attention network model
1) a generative adversarial network modeling method based on a convolutional multi-head attention mechanism for aero-engine missing data is mainly composed of a generator G and a discriminator D; the generator G is composed of a parallel convolutional layer, a fully connected layer, a position encoding layer, an N-layer TransformerEncoder module, another parallel convolutional layer and another fully connected layer, and is represented by the following formula:

Conv
⁢
1
⁢

d

1
×
1

&

⁢

Conv
⁢
1
⁢

d

1
×
3

-
Linear
-
PositionalEncoding
-

N
×
TransformerEncoder

-

Conv
⁢
1
⁢

d

1
×
1

*

Conv
⁢
1
⁢

d

1
×
3

-
Linear

(
2
)

2) constructing a random matrix Z equal to X in size, filling in a random number with a mean value of 0 and a variance of 0.1 for missing item data, and filling in 0 for unmissing item data; introducing a random value to make subsequent model training more robust;
constructing a matrix M′ which is identical to M according to the mask matrix M, and then setting all 0 terms in M′ to 1 with a probability of 90% to finally obtain a hint matrix H;
as input data of the generator G includes the standardized multivariate time series data X′, the random matrix Z, the mask matrix M and a pre-imputation matrix X pre , using the parallel convolutional layers to extract correlation information between the attributes, using position codes to encode time series information of the input data, using the N-layer TransformerEncoder module to effectively extract the time series information, using the parallel convolutional layers and the fully connected layers to output complete data information X g , and using X g to impute the missing item in X′; the discriminator D is similar to the generator G in structure, a Sigmoid activation function is only added

Assignees

• Univ Dalian Tech

Inventors

• Ma Song
• Sun Tao
• Xu Zengsong
• Sun Ximing
• Li Zhi

Classifications

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0475 Primary

  Generative networks · CPC title