System and method for management of inference models based on training data impact on model reversion

Patent metadata

Field	Value
Publication number	US-12306936-B2
Application number	US-202318193810-A
Country	US
Kind code	B2
Filing date	Mar 31, 2023
Priority date	Mar 31, 2023
Publication date	May 20, 2025
Grant date	May 20, 2025

Abstract

Methods and systems for managing inference models are disclosed. The inference models may be used to provide computer implemented services by generating inferences used in the services. The inference models may be managed by reverting inference models that are found to be compromised through training with poisoned training data. The type of reversion and training data to be used may be selected based on the cost for performing the reversion and benefits provided by the reverted inference model using graphical user interfaces.

First claim

What is claimed is:

1. A method for managing inference models, the method comprising:
identifying an inference model of the inference models that is tainted through training using poisoned training data;
identifying a first resource cost for reverting the tainted inference model to remove influence of the poisoned training data on the tainted inference model;
obtaining, using a graphical user interface, user input indicating a selection of a portion of the poisoned training data;
identifying a second resource cost for reverting the tainted inference model to remove influence of the portion of the poisoned training data on the tainted inference model;
obtaining a reversion plan for the tainted inference model based on the second resource cost;
performing the reversion plan to obtain an updated inference model; and
using the updated inference model to provide computer implemented services.

2. The method of claim 1 , wherein obtaining the user input comprises:
presenting, to a user, the graphical user interface comprising:
a range bar,
training data portion indicators, and
data selection indicators that discriminate a portion of the training data portion indicators, the portion of the training data portion indicators being associated with the portion of the poisoned training data; and

obtaining, from the user via the graphical user interface, the user input to position the data selection indicators along the range bar, the position of the data selection indicators defining the portion of the training data portion indicators.

3. The method of claim 2 , wherein obtaining the user input comprising:
reducing a dimensionality of the poisoned training data to obtain locations for the training data portion indicators; and
placing each training data portion indicator at a corresponding location of the locations, the locations being relative to the range bar.

4. The method of claim 3 , wherein the range bar defines a first axis of a two dimensional plot, with the first axis representing time progression and a second axis of the two dimensional plot representing the reduced dimensionality of the poisoned training data.

5. The method of claim 4 , wherein each portion of the poisoned training data is multidimensional, and reducing the dimensionality of each portion of the poisoned training data provides a single magnitude for the respective portion of the poisoned training data.

6. The method of claim 5 , wherein each of the training data portion indicators is positioned:
along the range bar based on when a corresponding portion of the poisoned training data was obtained, and
a distance away from the range bar based on the single magnitude for the corresponding portion of the poisoned training data.

7. The method of claim 2 , wherein the second resource cost is less than the first resource cost.

8. The method of claim 7 , wherein performing the reversion plan to obtain the updated inference model comprises:
training the tainted inference model based on portion of the poisoned training data to reduce the predictive power of the tainted inference model for relationships defined by the portion of the poisoned training data.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing inference models, the operations comprising:
identifying an inference model of the inference models that is tainted through training using poisoned training data;
identifying a first resource cost for reverting the tainted inference model to remove influence of the poisoned training data on the tainted inference model;
obtaining, using a graphical user interface, user input indicating a selection of a portion of the poisoned training data;
identifying a second resource cost for reverting the tainted inference model to remove influence of the portion of the poisoned training data on the tainted inference model;
obtaining a reversion plan for the tainted inference model based on the second resource cost;
performing the reversion plan to obtain an updated inference model; and
using the updated inference model to provide computer implemented services.

10. The non-transitory machine-readable medium of claim 9 , wherein obtaining the user input comprises:
presenting, to a user, the graphical user interface comprising:
a range bar,
training data portion indicators, and
data selection indicators that discriminate a portion of the training data portion indicators, the portion of the training data portion indicators being associated with the portion of the poisoned training data; and

obtaining, from the user via the graphical user interface, the user input to position the data selection indicators along the range bar, the position of the data selection indicators defining the portion of the training data portion indicators.

11. The non-transitory machine-readable medium of claim 10 , wherein obtaining the user input comprising:
reducing a dimensionality of the poisoned training data to obtain locations for the training data portion indicators; and
placing each training data portion indicator at a corresponding location of the locations, the locations being relative to the range bar.

12. The non-transitory machine-readable medium of claim 11 , wherein the range bar defines a first axis of a two dimensional plot, with the first axis representing time progression and a second axis of the two dimensional plot representing the reduced dimensionality of the poisoned training data.

13. The non-transitory machine-readable medium of claim 12 , wherein each portion of the poisoned training data is multidimensional, and reducing the dimensionality of each portion of the poisoned training data provides a single magnitude for the respective portion of the poisoned training data.

14. The non-transitory machine-readable medium of claim 13 , wherein each of the training data portion indicators is positioned:
along the range bar based on when a corresponding portion of the poisoned training data was obtained, and
a distance away from the range bar based on the single magnitude for the corresponding portion of the poisoned training data.

15. The non-transitory machine-readable medium of claim 9 , wherein the second resource cost is less than the first resource cost.

16. The non-transitory machine-readable medium of claim 15 , wherein performing the reversion plan to obtain the updated inference model comprises:
training the tainted inference model based on portion of the poisoned training data to reduce the predictive power of the tainted inference model for relationships defined by the portion of the poisoned training data.

17. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing inference models, the operations comprising:
identifying an inference model of the inference models that is tainted through training using poisoned training data;
identifying a first resource cost for reverting the tainted inference model to remove influence of the poisoned training data on the tainted inference model;
obtaining, using a graphical user interface, user input indicating a selection of a portion of the poisoned training data;
identifying a second resource cost for reverting the tainted inference model to remove influence of the portion of the poisoned training data on the tainted inference model;
obtaining a reversion plan for the tain

Assignees

• Dell Products Lp

Inventors

• Ezrielev Ofir
• Kushnir Tomer
• Savir Amihai

Classifications

• G06F21/55 Primary

  Detecting local intrusion or implementing counter-measures · CPC title