Method and Apparatus for Monitoring Machine Learning Models

Patent metadata

Field	Value
Publication number	US-2023016668-A1
Application number	US-202217954485-A
Country	US
Kind code	A1
Filing date	Sep 28, 2022
Priority date	Mar 31, 2020
Publication date	Jan 19, 2023
Grant date	—

Abstract

A method includes training a first control model by utilizing a first set of input data as first input, resulting in a trained first control model; copying the trained first control model to a second control model, wherein, after copying, the second input layer and the plurality of second hidden layers is identical to the plurality of first hidden layers, and the first output layer is replaced by the second output layer; freezing the plurality of second hidden layers; training the second control model by utilizing the first set of input data as second input, resulting in a trained second control model; and running the trained second control model by utilizing a second set of input data as second input, wherein the second output outputs the quality measure of the first control model.

First claim

What is claimed is:

1 . A method for determining a quality measure of a first control model for monitoring or controlling an industrial process, wherein the first control model is an artificial neural net, ANN, comprising a first input, a first input layer, a plurality of first hidden layers, a first output layer, and a first output, the method comprising:
training the first control model by utilizing a first set of input data as first input, resulting in a trained first control model; copying the trained first control model to a second control model, wherein the second control model comprises a second input, a second input layer, a plurality of second hidden layers, a second output layer, and a second output that is configured to output the quality measure of the first control model; wherein, after copying, the second input layer is identical to the first input layer, the plurality of second hidden layers is identical to the plurality of first hidden layers, and the first output layer is replaced by the second output layer; freezing at least parts of the plurality of second hidden layers; training the second control model by utilizing the first set of input data as a second input, resulting in a trained second control model; and running the trained second control model by utilizing a second set of input data as the second input, wherein the second output outputs the quality measure of the first control model.

2 . The method of claim 1 , wherein running the second control model is repeated at least one of periodically and on request.

3 . The method of claim 1 , further comprising:
comparing the quality measure to a predefined measure; and when the quality measure is determined to be outside the predefined measure, performing a predefined action.

4 . The method of claim 3 , wherein the predefined action comprises at least one of outputting an alarm, and re-training the first control model.

5 . The method of claim 4 , wherein re-training the first control model comprises:
unfreezing the hidden layers of the trained second control model; training the second control model by utilizing a third set of input data as the second input, wherein the third set of input data is a set of historical data selected from a plurality of second sets of input data, and wherein the second output outputs the quality measure; freezing the hidden layers of the second control model; comparing the quality measure to a predefined measure; and when the quality measure is determined to be inside the predefined measure, training the first control model by utilizing the third set of input data as the first input, resulting in a corrected trained first control model.

6 . The method of claim 5 , wherein when the quality measure is determined to be outside the predefined measure, the method further comprises repeating the steps of unfreezing, training, freezing, and comparing.

7 . The method of claim 1 , wherein training and/or re-training the first control model and/or the second control model comprises making and/or using predictions.

8 . A computer program product comprising computer executable instructions stored in a tangible medium that, when executed by a computer and/or an artificial neural net, ANN, cause the computer to and/or the ANN to determine a quality measure of a first control model for monitoring or controlling an industrial process, wherein the first control model comprises a first input, a first input layer, a plurality of first hidden layers, a first output layer, and a first output, the determining of the quality measure comprising:
training the first control model by utilizing a first set of input data as first input, resulting in a trained first control model; copying the trained first control model to a second control model, wherein the second control model comprises a second input, a second input layer, a plurality of second hidden layers, a second output layer, and a second output that is configured to output the quality measure of the first control model; wherein, after copying, the second input layer is identical to the first input layer, the plurality of second hidden layers is identical to the plurality of first hidden layers, and the first output layer is replaced by the second output layer; freezing at least parts of the plurality of second hidden layers; training the second control model by utilizing the first set of input data as a second input, resulting in a trained second control model; and running the trained second control model by utilizing a second set of input data as the second input, wherein the second output outputs the quality measure of the first control model.

9 . The computer program of claim 8 , wherein running the trained second control model is repeated at least one of periodically and on request.

10 . The computer program of claim 8 , further comprising:
comparing the quality measure to a predefined measure; and when the quality measure is determined to be outside the predefined measure, performing a predefined action.

11 . The computer program of claim 10 , wherein the predefined action comprises at least one of outputting an alarm, and re-training the first control model.

12 . The computer program of claim 11 , wherein re-training the first control model comprises:
unfreezing the hidden layers of the trained second control model; training the second control model by utilizing a third set of input data as the second input, wherein the third set of input data is a set of historical data selected from a plurality of second sets of input data, and wherein the second output outputs the quality measure; freezing the hidden layers of the second control model; comparing the quality measure to a predefined measure; and when the quality measure is determined to be inside the predefined measure, training the first control model by utilizing the third set of input data as the first input, resulting in a corrected trained first control model.

13 . The computer program of claim 12 , wherein when the quality measure is determined to be outside the predefined measure, the method further comprises repeating the steps of unfreezing, training, freezing, and comparing.

14 . The computer program of claim 8 , wherein training and/or re-training the first control model and/or the second control model comprises making and/or using predictions.

Assignees

• Abb Schweiz Ag

Inventors

• Schmidt Benedikt
• Amihai Ido
• Chioua Moncef
• Kotriwala Arzam
• Hollender Martin
• Janka Dennis
• Lenders Felix
• Schlake Jan Christoph
• Kloepper Benjamin
• Abukwaik Hadil

Classifications

• G06N3/088

  Non-supervised learning, e.g. competitive learning · CPC title

• G05B13/027 Primary

  using neural networks only · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G05B17/02

  electric · CPC title

• G05B23/024 Primary

  Quantitative history assessment, e.g. mathematical relationships between available data; Functions therefor; Principal component analysis [PCA]; Partial least square [PLS]; Statistical classifiers, e.g. Bayesian networks, linear regression or correlation analysis; Neural networks · CPC title