Framework for certifying a lower bound on a robustness level of convolutional neural networks

Patent metadata

Field	Value
Publication number	US-11625487-B2
Application number	US-201916256267-A
Country	US
Kind code	B2
Filing date	Jan 24, 2019
Priority date	Jan 24, 2019
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

A certification method, system, and computer program product include certifying an adversarial robustness of a convolutional neural network by deriving an analytic solution for a neural network output using an efficient upper bound and an efficient lower bound on an activation function and applying the analytic solution in computing a certified robustness.

First claim

What is claimed is:

1. A computer-implemented certification method, the method comprising:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,

wherein the efficient bound is derived using convolution operations.

2. The method of claim 1 , further comprising varying the activation function.

3. The method of claim 2 , wherein the activation function is varied until a numerical level of robustness of the neural network is within a predetermined threshold value.

4. The method of claim 1 , further comprising varying building blocks in the neural network.

5. The method of claim 4 , wherein the activation function is varied until a numerical level of robustness of the neural network is within a predetermined threshold value.

6. The method of claim 1 , further comprising varying both of the activation function and building blocks in the neural network.

7. The method of claim 1 , wherein the analytic solution is applied with a binary search.

8. The method of claim 1 , wherein an efficient upper bound as one of the efficient bound comprises a linear upper bound, and
wherein an efficient lower bound as one of the efficient bound comprises a linear lower bound.

9. The method of claim 1 , wherein the adversarial robustness is certified for a same input.

10. The method of claim 1 , embodied in a cloud-computing environment.

11. The method of claim 1 , further comprising computing the numerical level of the robustness a specific architecture of the CNNs based on the analytic solution,
wherein the analytic solution includes deriving, for each building block in the form of element-wise inequality equations, and then plugging in the corresponding bounds and back-propagate to a previous layer of the CNNs.

12. A computer program product for certification, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,

wherein the efficient bound is derived using convolution operations.

13. The computer program product of claim 12 , further comprising varying the activation function.

14. The computer program product of claim 13 , wherein the activation function is varied until a certified robustness of the neural network is within a predetermined threshold value.

15. The computer program product of claim 12 , further comprising varying building blocks in the neural network.

16. The computer program product of claim 15 , wherein the activation function is varied until a certified robustness of the neural network is within a predetermined threshold value.

17. The computer program product of claim 12 , further comprising varying both of the activation function and building blocks in the neural network.

18. The computer program product of claim 12 , wherein the analytic solution is applied with a binary search.

19. The computer program product of claim 12 , wherein an efficient upper bound as one of the efficient bound comprises a linear upper bound, and
wherein an efficient lower bound as one of the efficient bound comprises a linear lower bound.

20. A certification system, the system comprising:
a processor, and
a memory, the memory storing instructions to cause the processor to perform:
certifying a numerical level of robustness of various architectures of convolutional neutral networks (CNNs) with minimum adversarial distortion by:
deriving an analytic solution for a neural network output of the CNNs using an efficient block-wise linear bound on an activation function separately on non-linear operations in the CNNs,

wherein the efficient bound is derived using convolution operations.

Assignees

• IBM
• Massachusetts Inst Technology

Inventors

• Chen Pin-Yu
• Liu Sijia
• Boopathy Akhilan
• Weng Tsui-Wei
• Daniel Luca

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title