Risk evaluation apparatus, risk evaluation method, and non-transitory computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-12292976-B2
Application number	US-202017782195-A
Country	US
Kind code	B2
Filing date	Jan 14, 2020
Priority date	Jan 14, 2020
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

The risk evaluation apparatus evaluates the risk of a machine learning model. The risk evaluation apparatus includes a recording unit, a loss function regression model acquirer, an attack noise addition unit, an error acquisition unit, and an evaluation unit. The recording unit records a set of predetermined loss functions and a set of pairs of data and labels predetermined. The loss function regression model acquirer determines a regression model of the loss function in the vicinity of data by nonparametric regression. The attack noise addition unit creates attack data that is an Adversarial Example using the regression model. The error acquisition unit determines the error between the output of the machine learning model when the data is input and the output of the machine learning model when the attack data is input. The evaluation unit evaluates the risk based on a set of errors.

First claim

The invention claimed is:

1. A risk evaluation apparatus for evaluating a risk of a machine learning model, the risk evaluation apparatus comprising:
a recording medium configured to record a set of loss functions that are predetermined and a set of a plurality of pairs of data and labels that are predetermined; and
processing circuitry configured to:
execute a loss function regression model acquirer process which determines, for each of the loss functions and for each of the plurality of pairs of data and labels, a regression model of the loss function in a vicinity of the data by nonparametric regression;
execute an attack noise addition process which creates attack data using the regression model for each of the loss functions and for each of the plurality of pairs of data and labels;
execute an error acquisition process which obtains a set of errors, each of which is an error between an output of the machine learning model in a case where the data recorded in the recording medium is input and an output of the machine learning model in a case where the attack data is input for each of the loss functions and for each of the plurality of pairs of data and labels; and
execute an evaluation process which evaluates a risk based on the set of errors.

2. The risk evaluation apparatus according to claim 1 , wherein
the loss function regression model acquirer process:
randomly changes the data in a vicinity of the data in accordance with a predetermined distribution within a space that can be input into the machine learning model, to determine random change data;
determines regression training data, the regression training data being a pair of the random change data and a loss value obtained from the loss function in a case where an output of the machine learning model when the random change data is input and the label are input; and
determines the regression model using a set of the regression training data.

3. The risk evaluation apparatus according to claim 2 , wherein
the random change data is obtained by:
adding, to the data, a random number generated in accordance with the predetermined distribution and performing adjustment for obtaining an addition result being within a space that can be input into the machine learning model.

4. The risk evaluation apparatus according to claim 2 , wherein
the predetermined distribution is a normal distribution.

5. The risk evaluation apparatus according to claim 1 , wherein
the nonparametric regression is a Gaussian process regression.

6. A risk evaluation method for evaluating a risk of a machine learning model, the risk evaluation method comprising:
preparing a set of loss functions that are predetermined and a set of a plurality of pairs of data and labels that are predetermined;
determining, for each of the loss functions and for each of the plurality of pairs of data and labels, a regression model of the loss function in a vicinity of the data by nonparametric regression;
creating, for each of the loss functions and for each of the plurality of pairs of data and labels, attack data being an Adversarial Example using the regression model;
obtaining a set of errors, each of which is an error between an output of the machine learning model in a case where the data prepared is input for each of the loss functions and for each of the plurality of pairs of data and labels; and
evaluating a risk based on the set of errors.

7. The risk evaluation apparatus according to claim 3 , wherein
the predetermined distribution is a normal distribution.

8. The risk evaluation apparatus according to claim 2 , wherein
the nonparametric regression is a Gaussian process regression.

9. The risk evaluation apparatus according to claim 3 , wherein
the nonparametric regression is a Gaussian process regression.

10. The risk evaluation apparatus according to claim 4 , wherein
the nonparametric regression is a Gaussian process regression.

11. A non-transitory computer-readable recording medium storing executable instructions thereon which, when executed by circuitry, cause the executable instructions to perform a method for evaluating a risk of a machine learning model, the method comprising:
recording a set of loss functions that are predetermined and a set of a plurality of pairs of data and labels that are predetermined;
executing a loss function regression model acquirer process which determines, for each of the loss functions and for each of the plurality of pairs of data and labels, a regression model of the loss function in a vicinity of the data by nonparametric regression;
executing an attack noise addition process which creates attack data using the regression model for each of the loss functions and for each of the plurality of pairs of data and labels;
executing an error acquisition process which obtains a set of errors, each of which is an error between an output of the machine learning model in a case where the data recorded in the recording medium is input and an output of the machine learning model in a case where the attack data is input for each of the loss functions and for each of the plurality of pairs of data and labels; and
executing an evaluation process which evaluates a risk based on the set of errors.

Assignees

• Nippon Telegraph & Telephone

Inventors

• Okada Rina
• Hasegawa Satoshi

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06F21/57 Primary

  Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title