Inline detection and prevention of adversarial attacks

Patent metadata

Field	Value
Publication number	US-2022156376-A1
Application number	US-202016952494-A
Country	US
Kind code	A1
Filing date	Nov 19, 2020
Priority date	Nov 19, 2020
Publication date	May 19, 2022
Grant date	—

Abstract

A processor may generate an enforcement point. The enforcement point may include one or more adversarial detection models. The processor may receive user input data. The processor may analyze, at the enforcement point, the user input data. The processor may determine, from the analyzing, whether there is an adversarial attack in the user input data. The processor may generate an alert based on the determining.

First claim

What is claimed is:

1 . A method for inline detection and prevention of adversarial attacks, the method comprising:
generating, by a processor, an enforcement point, wherein the enforcement point includes one or more adversarial detection models; receiving user input data; analyzing, at the enforcement point, the user input data; determining, from the analyzing, whether there is an adversarial attack in the user input data; and generating an alert based on the determining.

2 . The method of claim 1 , wherein the enforcement point is in communication with a machine learning framework, and wherein the machine learning framework includes updated information on identified adversarial attacks based on machine learning models.

3 . The method of claim 2 , further comprising:
updating information at the enforcement point in regard to the updated information on identified adversarial attacks; and forwarding the updated information to one or more machine learning applications.

4 . The method of claim 2 , wherein analyzing the user input data includes:
analyzing the user input data in regard to the updated information on identified adversarial attacks.

5 . The method of claim 4 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, there is no adversary; and forwarding the user input data to one or more machine learning applications, wherein the forwarding of the user input data includes metadata that indicates a level of confidence in regard to the adversary.

6 . The method of claim 4 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, a confirmed adversary; and stopping the forwarding of the user input data to one or more machine learning applications.

7 . The method of claim 6 , further comprising:
forwarding the user input data to the machine learning framework.

8 . A system comprising:
a memory; and a processor in communication with the memory, the processor being configured to perform operations comprising: generating an enforcement point, wherein the enforcement point includes one or more adversarial detection models; receiving user input data; analyzing, at the enforcement point, the user input data; determining, from the analyzing, whether there is an adversarial attack in the user input data; and generating an alert based on the determining.

9 . The system of claim 8 , wherein the enforcement point is in communication with a machine learning framework, and wherein the machine learning framework includes updated information on identified adversarial attacks based on machine learning models.

10 . The system of claim 9 , the processor being further configured to perform operations comprising:
updating information at the enforcement point in regard to the updated information on identified adversarial attacks; and forwarding the updated information to one or more machine learning applications.

11 . The system of claim 9 , wherein analyzing the user input data includes:
analyzing the user input data in regard to the updated information on identified adversarial attacks.

12 . The system of claim 11 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, there is no adversary; and forwarding the user input data to one or more machine learning applications, wherein the forwarding of the user input data includes metadata that indicates a level of confidence in regard to the adversary.

13 . The system of claim 11 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, a confirmed adversary; and stopping the forwarding of the user input data to one or more machine learning applications.

14 . The system of claim 13 , the processor being further configured to perform operations comprising:
forwarding the user input data to the machine learning framework.

15 . A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations, the operations comprising:
generating an enforcement point, wherein the enforcement point includes one or more adversarial detection models; receiving user input data; analyzing, at the enforcement point, the user input data; determining, from the analyzing, whether there is an adversarial attack in the user input data; and generating an alert based on the determining.

16 . The computer program product of claim 15 , wherein the enforcement point is in communication with a machine learning framework, and wherein the machine learning framework includes updated information on identified adversarial attacks based on machine learning models.

17 . The computer program product of claim 16 , the operations further comprising:
updating information at the enforcement point in regard to the updated information on identified adversarial attacks; and forwarding the updated information to one or more machine learning applications.

18 . The computer program product of claim 16 , wherein analyzing the user input data includes:
analyzing the user input data in regard to the updated information on identified adversarial attacks.

19 . The computer program product of claim 18 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, there is no adversary; and forwarding the user input data to one or more machine learning applications, wherein the forwarding of the user input data includes metadata that indicates a level of confidence in regard to the adversary.

20 . The computer program product of claim 18 , wherein determining whether there is an adversarial attack in the user input data further includes:
identifying, from the updated information on identified adversarial attacks, a confirmed adversary; and stopping the forwarding of the user input data to one or more machine learning applications.

Assignees

• IBM

Inventors

• Dos Santos Silva Bruno
• Lee Cheng-Ta
• Williams Ron
• Kuo Bo-Yu
• Chang Chao-Min
• Muppidi Sridhar

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title