Method, apparatus and computer program for generating robust automated learning systems and testing trained automated learning systems

Patent metadata

Field	Value
Publication number	US-11386328-B2
Application number	US-201816173698-A
Country	US
Kind code	B2
Filing date	Oct 29, 2018
Priority date	May 30, 2018
Publication date	Jul 12, 2022
Grant date	Jul 12, 2022

Abstract

In a method for training a first neural network a superposed classification is back-propagated through a second neural network. An output value of the second neural network is utilized to determine whether the input of the first neural network is adversarial.

First claim

What is claimed is:

1. A method for testing a first automated learning system, the first automated learning system being a neural network having (i) an input layer configured to receive a test input value, (ii) hidden layers, and (iii) an output layer configured to output a first output value, the method comprising:
determining a second output value based on (i) a predefined target output value that is different than a correct output value that is assigned to the test input value and (ii) a reference output value that is one of the correct output value and the first output value of the first automated learning system;
propagating the second output value through a second automated learning system, the second automated learning system being a neural network having (i) an input layer configured to perform a transformation that is a conjugate of a transformation performed by the output layer of the first automated learning system and being configured to receive the second output value, (ii) hidden layers each corresponding to a respective layer of the hidden layers of the first automated learning system but being arranged in reverse order compared to the corresponding hidden layers of the first automated learning system, each of the hidden layers of the second automated learning system being configured to perform a transformation that is a conjugate of a transformation performed by the corresponding respective layer of the hidden layers of the first automated learning system, and (iii) an output layer configured to perform a transformation that is a conjugate of a transformation performed by the input layer of the first automated learning system and being configured to output a third output value; and
determining, based on the third output value that results from the propagation of the second output value through the second automated learning system, whether modifications to the test input value cause the output layer of the first automated learning system to output the predefined target output value, the modifications being less than or equal to a predetermined modification magnitude.

2. The method according to claim 1 , further comprising:
issuing a robustness certificate in response to determining that the modifications to the test input value do not cause the output layer of the first automated learning system to output the predefined target output value.

3. The method according to claim 2 , the method further comprising:
controlling a physical actuator of a technical system depending on further output values of the first automated learning system and depending on the issued robustness certificate.

4. The method according to claim 1 , further comprising:
determining an objective function based on (i) the third output value of the second automated learning system, (ii) the modification magnitude, and (iii) the test input value;
comparing the determined objective function to a predetermined threshold; and
determining whether the modifications to the test input value cause the output layer of the first automated learning system to output the predefined target output value depending on the result of a comparison.

5. A method for determining a largest modification magnitude of a modification to a test input value to an input layer of a first automated learning system that does not cause an output layer of the first automated learning system to output a predefined target output value that is different than a correct output value that is assigned to the test input value, the first automated learning system being a neural network having (i) the input layer, (ii) hidden layers, and (iii) the output layer, the method comprising:
determining a second output value based on (i) the predefined target output value and (ii) a reference output value that is one of the correct output value and a first output value of the first automated learning system;
propagating the second output value through a second automated learning system, the second automated learning system being a neural network having (i) an input layer configured to perform a transformation that is a conjugate of a transformation performed by the output layer of the first automated learning system and being configured to receive the second output value, (ii) hidden layers each corresponding to a respective layer of the hidden layers of the first automated learning system but being arranged in reverse order compared to the corresponding hidden layers of the first automated learning system, each of the hidden layers of the second automated learning system being configured to perform a transformation that is a conjugate of a transformation performed by the corresponding respective layer of the hidden layers of the first automated learning system, and (iii) an output layer configured to perform a transformation that is a conjugate of a transformation performed by the input layer of the first automated learning system and being configured to output a third output value;
determining an objective function based on (i) the test input value, (ii) the third output value that results from the propagation of the second output value through the second automated learning system, and (iii) the modification magnitude; and
determining the largest modification magnitude depending on the objective function such that the objective function does not become smaller than a predetermined threshold.

6. The method according to claim 5 , wherein the predetermined threshold for the objective function is not less than zero.

7. The method according to claim 5 , wherein:
one of a plurality of second output values is determined for each one of a plurality of predefined target output values,
each one of the plurality of predefined target output values corresponds to a differing output value that is different from the reference output value,
the plurality of second output values are propagated through the second automated learning system which outputs a plurality of corresponding output values, and
the objective function is determined depending on said plurality of corresponding output values.

8. The method according to claim 5 , further comprising:
detecting whether the test input value of the first automated learning system is anomalous by determining, based on the third output value that results from the propagation of the second output value through the second automated learning system, whether modifications to the test input value cause the output layer of the first automated learning system to output the predefined target output value, the modifications being less than or equal to a predetermined modification magnitude,
wherein the predetermined modification magnitude is selected not to be greater than the largest modification magnitude.

9. A method for training a first automated learning system, the first automated learning system being a neural network having (i) an input layer, (ii) hidden layers, and (iii) an output layer, the method comprising:
providing (i) a predetermined modification magnitude and (ii) training data including both training input values and corresponding training output values;
providing a second automated learning system based on the automated learning system, the second automated learning system being a neural network having (i) an input layer configured to perform a respective second transformation that is a conjugate of a respective first transformation performed by the output layer of the first automated learning system, (ii) hidden layers each corresponding to a respective layer of the hidden layers of the first automated learning system but being arranged in reverse order compared to the corresponding hidden layers of the first automated learning system, each of the hidden laye

Assignees

• Bosch Gmbh Robert
• Univ Carnegie Mellon

Inventors

• Wong Eric
• Schmidt Frank
• Hendrick Metzen Jan
• Zico Kolter Jeremy

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G05B13/042 Primary

  in which a parameter or coefficient is automatically adjusted to optimise the performance · CPC title