Method and machine learning system for detecting adversarial examples

Patent metadata

Field	Value
Publication number	US-2021089957-A1
Application number	US-201916576830-A
Country	US
Kind code	A1
Filing date	Sep 20, 2019
Priority date	Sep 20, 2019
Publication date	Mar 25, 2021
Grant date	—

Abstract

A method and machine learning system for detecting adversarial examples is provided. A first machine learning model is trained with a first machine learning training data set having only training data samples with robust features. A second machine learning model is trained with a second machine learning training data set, the second machine learning training data set having only training data samples with non-robust features. A feature is a distinguishing element in a data sample. A robust feature is more resistant to adversarial perturbations than a non-robust feature. A data sample is provided to each of the first and second trained machine learning models during an inference operation. if the first trained machine learning model classifies the data sample with high confidence, and the second trained machine learning model classifies the data sample differently with a high confidence, then the data sample is determined to be an adversarial example.

First claim

What is claimed is:

1 . A method for detecting adversarial examples, the method comprising:
training a first machine learning model with a first machine learning training data set having only training data samples with robust features, to produce a first trained machine learning model; training a second machine learning model with a second machine learning training data set, the second machine learning training data set having only training data samples with non-robust features to produce a second trained machine learning model, wherein a feature is a distinguishing element in a data sample, and wherein a robust feature is more resistant to adversarial perturbations than a non-robust feature; and providing a data sample to each of the first and second trained machine learning models during an inference operation, if the first trained machine learning model classifies the data sample with high confidence, and the second trained machine learning model classifies the data sample differently with a high confidence, then the data sample is determined to be an adversarial example.

2 . The method of claim 1 , wherein the first and second machine learning models include the same machine learning algorithm.

3 . The method of claim 1 , wherein the first and second machine learning models are based on a neural network.

4 . The method of claim 1 , wherein if the first and second trained machine learning models classify the data sample the same, the data sample is determined to not be an adversarial example.

5 . The method of claim 1 , further comprising training a third machine learning model with a third training data set, the third training data set not having any protections against adversarial examples.

6 . The method of claim 5 , further comprising providing the data sample to the third trained machine learning model if the data sample is determined not to be an adversarial example.

7 . The method of claim 1 , wherein the data sample is an image having a non-robust feature, the non-robust feature being imperceptible by a human being.

8 . A method for detecting adversarial examples, the method comprising:
compiling a set of robust features and a set of non-robust features, wherein a feature is a distinguishing element in a data sample, and wherein a robust feature is more resistant to adversarial perturbations than a non-robust feature; creating a first machine learning training data set having only training data samples with the robust features; creating a second machine learning training data set having only training data samples with the non-robust features; training a first machine learning model with the first machine learning training data set to produce a first trained machine learning model; training a second machine learning model with the second machine learning training data set to produce a second trained machine learning model; and providing a data sample to each of the first and second trained machine learning models during an inference operation, if the first trained machine learning model classifies the data sample with high confidence, and the second trained machine learning model classifies the data sample differently with high confidence, the data sample is determined to be an adversarial example.

9 . The method of claim 8 , wherein if the first trained machine learning model and the second trained machine learning model classify the data sample the same, the data sample is determined to not be an adversarial example.

10 . The method of claim 9 , wherein the first and second trained machine learning models both include the same machine learning algorithm.

11 . The method of claim 10 , further comprising providing the data sample that is determined to not be an adversarial example to a third trained machine learning model that has been trained without any protections against adversarial examples.

12 . The method of claim 8 , wherein the first, second, and third machine learning models all include a machine learning algorithm for classifying images.

13 . The method of claim 8 , further comprising providing an indication of an attack in response to the adversarial example being detected.

14 . The method of claim 8 , wherein the first, second, and third machine learning models all include a neural network.

15 . A machine learning system comprising:
a first trained machine learning model trained with a first training data set including only a plurality of robust features, the first trained machine learning model having an input for receiving an input data sample, and an output for providing a first output classification in response to receiving the input data sample; a second trained machine learning model trained with a second training data set, the second training data set including only a plurality of non-robust features, the second trained machine learning model having an output for providing a second output classification in response to receiving the input data sample, wherein a feature is characterized as being a distinguishing element of a data sample, and wherein a robust feature is more resistant to adversarial perturbations than a non-robust feature; and a distinguisher coupled to an output of both the first and second trained machine learning models for receiving the first and second output classifications, if the first trained machine learning model classifies the data sample with high confidence, and the second trained machine learning model classifies the data sample differently than the first trained machine learning model and with high confidence, the data sample is determined to be an adversarial example.

16 . The machine learning system of claim 15 , wherein if the first and second trained machine learning models classify the data sample the same, the data sample is determined to not be an adversarial example.

17 . The machine learning system of claim 15 , further comprising a third trained machine learning model trained with a third training data set, wherein the third training data set not trained to have any protections against adversarial examples.

18 . The machine learning model of claim 17 , wherein if the first and second trained machine learning models classify the data sample the same, the data sample is determined to not be an adversarial example and the data sample is provided to the third trained machine learning model for classification.

19 . The machine learning model of claim 15 , wherein the first and second trained machine learning models both use the same machine learning algorithm.

20 . The machine learning model of claim 15 , wherein the first and second trained machine learning models include a neural network.

Assignees

• Nxp Bv

Inventors

• Ermans Brian
• Doliwa Peter
• Van Vredendaal Christine

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06N3/045

  Combinations of networks · CPC title