Testing adversarial robustness of systems with limited access

Patent metadata

Field	Value
Publication number	US-11836256-B2
Application number	US-201916256107-A
Country	US
Kind code	B2
Filing date	Jan 24, 2019
Priority date	Jan 24, 2019
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

An adversarial robustness testing method, system, and computer program product include testing a robustness of a black-box system under different access settings via an accelerator.

First claim

What is claimed is:

1. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,
wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and a legitimate image.

2. The method of claim 1 , wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.

3. The method of claim 1 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find the adversarial examples and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.

4. The method of claim 1 , further comprising, given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that the perturbed example is misclassified to a target class including an incorrect class label by a deep neural network (DNN) model trained on the legitimate inputs.

5. The method of claim 1 , wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.

6. The method of claim 1 , wherein the accelerator comprises a function including a dimension reduction of an input.

7. The method of claim 1 , wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.

8. The method of claim 1 , embodied in a cloud-computing environment.

9. A computer program product for adversarial robustness testing for checking a learning performance of a black-box system, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,
wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.

10. The computer program product of claim 9 , wherein the different access settings comprise:
a soft-label setting; and
a hard-label setting.

11. The computer program product of claim 9 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find adversarial example and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.

12. The computer program product of claim 9 , further comprising, given a legitimate input of a plurality of legitimate inputs having a correct class label, determining an optimal adversarial perturbation using the accelerator such that the perturbed example is misclassified to a target class including an incorrect class label by a deep neural network (DNN) model trained on the legitimate inputs.

13. The computer program product of claim 9 , wherein the accelerator comprises a function including an efficient gradient estimation via a random directional estimate and averaging.

14. The computer program product of claim 9 , wherein the accelerator comprises a function including a dimension reduction of an input.

15. The computer program product of claim 9 , wherein the accelerator comprises a function including a problem splitting between a black-box loss function and a white-box adversarial distortion function.

16. An adversarial robustness testing system for checking a learning performance of a black-box system, the system comprising:
a processor; and
a memory, the memory storing instructions to cause the processor to perform:
testing a robustness, against an adversarial attack, of a black-box system under different access settings via an accelerator by generating adversarial inputs of a model in a limited access setting of the different access settings,

wherein the accelerator includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.

17. The system of claim 16 , further comprising:
for a soft-label setting as one of the different access settings, using the accelerator and a gradient descent technique to find the adversarial examples and summarize a robustness statistic; and
for a hard-label setting as one of the different access settings, using a smoothing function to summarize a robustness statistic.

18. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
receiving a first classification of an input as an output from the black-box system; and
determining a minimal change to the input such that a second classification is received as the output from the black-box system,

wherein the testing includes a function to reduce an attack space in the adversarial attack for query efficiency,
wherein a robustness objective for the testing of the robustness of the black-box system uses system defined threat models for adversarial examples, and
wherein a perturbed noise at each pixel of the perturbed example is imperceptible up to a predefined ε-tolerant threshold and a non-negative regularization parameter places emphasis on a distortion between the adversarial examples and the legitimate image.

19. A computer-implemented adversarial robustness testing method for checking a learning performance of a black-box system, the method comprising:
testing a robustness, against an adversarial attack, of the black-box system under a limited access setting to the black-box system:
f

Assignees

• IBM

Inventors

• Chen Pin-Yu
• Liu Sijia
• Wu Lingfei
• Chen Chia-Yu

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title