Accelerated machine learning

Patent metadata

Field	Value
Publication number	US-2022092464-A1
Application number	US-202016948564-A
Country	US
Kind code	A1
Filing date	Sep 23, 2020
Priority date	Sep 23, 2020
Publication date	Mar 24, 2022
Grant date	—

Abstract

Various embodiments are provided for accelerating machine learning in a computing environment by one or more processors in a computing system. Selected data may be received for training machine learning pipelines. Each of the machine learning pipelines may be scored according to one or more learning curves while training on selected data. Completion of the training on the selected data may be permitted for those of the machine learning pipelines having a score greater than a selected threshold. The training on the selected data may be terminated, prior to completion, on those of the machine learning pipelines having a score less than a selected threshold.

First claim

What is claimed is:

1 . A method for automated evaluation of machine learning models in a computing environment by one or more processors comprising:
automatically evaluating and determining a level of robustness of a machine learning model against adversarial whitebox attacks; and analyzing results from the adversarial attack and generating a modified machine learning model in response to the evaluating and determining.

2 . The method of claim 1 , further including receiving the machine learning model, a data set used for testing the machine learning model, one or more adversarial attack objectives, an attack threat model, and a selected number of hyperparameters.

3 . The method of claim 1 , further including generating an evaluation summary based on evaluating and determining of the level of robustness of the machine learning model.

4 . The method of claim 1 , further including automatically commencing the evaluating and determining the level of robustness of the machine learning model against the adversarial whitebox attacks using partial inputs from a previous evaluation of the machine learning model.

5 . The method of claim 1 , further including:
adjusting one or more adversarial attack objectives, an attack threat model, a selected number of hyperparameters, and a data set used for testing the machine learning model; and reconfiguring or adjusting an unmasking of gradients of the machine learning model, a loss function, an adversarial attack, and reanalyzing the results from the adversarial attacks based.

6 . The method of claim 1 , further including:
determining a robustness score for the machine learning model indicating a level of security from against adversarial whitebox attacks; and ranking the machine learning model based on the robustness score.

7 . The method of claim 1 , further including initializing a machine learning operation to:
learn and store the level of robustness of the machine learning model against the adversarial whitebox attacks based on the machine learning model, a data set used for testing the machine learning model, one or more adversarial attack objectives, an attack threat model, and a selected number of hyperparameters; and collect feedback in relation to automatically performing the diagnosis and evaluation of the level of robustness of a machine learning model against adversarial whitebox attacks to generate the modified machine learning model in response to performing at least a portion of the diagnosis and evaluation operation.

8 . A system for automated evaluation of machine learning models in a computing environment, comprising:
one or more computers with executable instructions that when executed cause the system to:
automatically evaluate and determine a level of robustness of a machine learning model against adversarial whitebox attacks; and
analyzing results from the adversarial attack and generating a modified machine learning model in response to the evaluating and determining.

9 . The system of claim 8 , wherein the executable instructions when executed cause the system to receive the machine learning model, a data set used for testing the machine learning model, one or more adversarial attack objectives, an attack threat model, and a selected number of hyperparameters.

10 . The system of claim 8 , wherein the executable instructions when executed cause the system to generate an evaluation summary based on evaluating and determining of the level of robustness of the machine learning model.

11 . The system of claim 8 , wherein the executable instructions when executed cause the system to automatically commence evaluating and determining the level of robustness of the machine learning model against the adversarial whitebox attacks using partial inputs from a previous evaluation of the machine learning model.

12 . The system of claim 8 , wherein the executable instructions when executed cause the system to:
adjust one or more adversarial attack objectives, an attack threat model, a selected number of hyperparameters, and a data set used for testing the machine learning model; and reconfigure or adjust an unmasking of gradients of the machine learning model, a loss function, an adversarial attack, and reanalyzing the results from the adversarial attacks based.

13 . The system of claim 8 , wherein the executable instructions when executed cause the system to:
determine a robustness score for the machine learning model indicating a level of security from against adversarial whitebox attacks; and rank the machine learning model based on the robustness score.

14 . The system of claim 8 , wherein the executable instructions when executed cause the system to initialize a machine learning operation to:
learn and store the level of robustness of the machine learning model against the adversarial whitebox attacks based on the machine learning model, a data set used for testing the machine learning model, one or more adversarial attack objectives, an attack threat model, and a selected number of hyperparameters; and collect feedback in relation to automatically performing the diagnosis and evaluation of the level of robustness of a machine learning model against adversarial whitebox attacks to generate the modified machine learning model in response to performing at least a portion of the diagnosis and evaluation operation.

15 . A computer program product for automated evaluation of machine learning models in a computing environment, the computer program product comprising:
one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instruction comprising:
program instructions to automatically evaluate and determine a level of robustness of a machine learning model against adversarial whitebox attacks; and
analyze results from the adversarial attack and generate a modified machine learning model in response to the evaluating and determining.

16 . The computer program product of claim 15 , further including program instructions to receive the machine learning model, a data set used for testing the machine learning model, one or more adversarial attack objectives, an attack threat model, and a selected number of hyperparameters.

17 . The computer program product of claim 15 , further including program instructions to generate an evaluation summary based on evaluating and determining of the level of robustness of the machine learning model.

18 . The computer program product of claim 15 , further including program instructions to automatically commence evaluating and determining the level of robustness of the machine learning model against the adversarial whitebox attacks using partial inputs from a previous evaluation of the machine learning model.

19 . The computer program product of claim 15 , further including program instructions to:
adjust one or more adversarial attack objectives, an attack threat model, a selected number of hyperparameters, and a data set used for testing the machine learning model; and reconfigure or adjust an unmasking of gradients of the machine learning model, a loss function, an adversarial attack, and reanalyzing the results from the adversarial attacks based.

20 . The computer program product of claim 15 , further including program instructions to:
determine a robustness score for the machine learning model indicating a level of security from against adversarial whitebox attacks; and rank the

Assignees

• IBM

Inventors

• Wistuba Martin
• Pedapati Tejaswini

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title

• G06F2221/033

  Test or assess software · CPC title

• G06F21/554

  involving event detection and direct action · CPC title

• G06F2221/034

  Test or assess a computer or a system · CPC title