System and method for proactively identifying poisoned training data used to train artificial intelligence models

Patent metadata

Field	Value
Publication number	US-12481793-B2
Application number	US-202218147757-A
Country	US
Kind code	B2
Filing date	Dec 29, 2022
Priority date	Dec 29, 2022
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

Methods and systems for identifying poisoned training data used for training artificial intelligence (AI) models are disclosed. To identify poisoned training data in a proposed training dataset, a causal model may be obtained. The causal model may include relationships relating data elements. The proposed training dataset may be identified as poisoned when data elements within the proposed training dataset do not satisfy the relationships set forth by the causal model. When the identification of poisoned training data is made, the AI model may not be updated using the proposed training dataset and the proposed training dataset may be discarded. If poisoned training data is not identified prior to training an AI model, methods and systems are disclosed for the remediation of the poisoned training dataset and subsequent tainted AI models. By doing so, the effect of poisoned training data may be prevented and/or efficiently computationally mitigated.

First claim

What is claimed is:

1 . A method for identifying poisoned training data used for training an artificial intelligence (AI) model, comprising:
making an identification that a second training dataset usable to retrain a first instance of the AI model is available; and
determining, based on the identification and before using the second training dataset to retrain the first instance of the AI model, whether the second training dataset is poisoned by, at least:
identifying a causal relationship within a first training dataset, the first training dataset being used to train the first instance of the AI model, and
identifying a third variable and a fourth variable from a second plurality of variables of the second training dataset, the third variable and the fourth variable being analogous to a first variable and a second variable of the first training dataset, respectively; and
transforming data elements of the third variable based on the causal relationship to obtained transformed data elements of the third variable
comparing the transformed data elements of the third variable to data elements of the fourth variable to make a determination regarding whether the third variable and the fourth variable satisfy the causal relationship,
in a first instance of the determination where the third variable and the fourth variable satisfy the causal relationship:
treating the second training dataset as not being poisoned, and
retraining the first instance of AI model using the second training dataset to obtain a second instance of the of the AI model; and

in a second instance of the determination where the third variable and the fourth variable do not satisfy the causal relationship:
treating the second training dataset as being poisoned.

2 . The method of claim 1 , further comprising, after making the identification and before using the second training dataset to retrain the first instance of the AI model:
obtaining the first instance of the AI model; and obtaining a causal model based on the first training dataset, the causal model comprising the causal relationship.

3 . The method of claim 1 , wherein identifying the causal relationship within the first training dataset comprises:
identifying a first variable from a first plurality of variables of the first training dataset and a second variable from the first plurality of variables of the first training dataset; and reading the causal relationship from a causal model, the causal relationship defining a functional relationship between the first variable and the second variable.

4 . The method of claim 1 , wherein treating the second training dataset as being poisoned comprises passing on a retraining opportunity for the AI model presented by the second training dataset.

5 . The method of claim 4 , wherein passing on the retraining opportunity comprises discarding the second training dataset.

6 . The method of claim 1 , wherein the causal relationship is identified using a causal model comprising nodes and edges between the nodes, the nodes correspond to portions of the first training data set, and the edges between the nodes correspond to relationships between the portions of the first training data set corresponding to the nodes.

7 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for identifying poisoned training data used for training an artificial intelligence (AI) model, the operations comprising:
making an identification that a second training dataset usable to retrain a first instance of the AI model is available; and
determining, based on the identification and before using the second training dataset to retrain the first instance of the AI model, whether the second training dataset is poisoned by, at least:
identifying a causal relationship within a first training dataset, the first training dataset being used to train the first instance of the AI model, and
identifying a third variable and a fourth variable from a second plurality of variables of the second training dataset, the third variable and the fourth variable being analogous to a first variable and a second variable of the first training dataset, respectively; and
transforming data elements of the third variable based on the causal relationship to obtained transformed data elements of the third variable
comparing the transformed data elements of the third variable to data elements of the fourth variable to make a determination regarding whether the third variable and the fourth variable satisfy the causal relationship,
in a first instance of the determination where the third variable and the fourth variable satisfy the causal relationship:
treating the second training dataset as not being poisoned, and
retraining the first instance of AI model using the second training dataset to obtain a second instance of the of the AI model; and

in a second instance of the determination where the third variable and the fourth variable do not satisfy the causal relationship:
treating the second training dataset as being poisoned.

8 . The non-transitory machine-readable medium of claim 7 , the operations further comprising:
obtaining the first instance of the AI model; and obtaining a causal model based on the first training dataset, the causal model comprising the causal relationship.

9 . The non-transitory machine-readable medium of claim 7 , wherein identifying the causal relationship within the first training dataset comprises:
identifying a first variable from a first plurality of variables of the first training dataset and a second variable from the first plurality of variables of the first training dataset; and reading the causal relationship from a causal model, the causal relationship defining a functional relationship between the first variable and the second variable.

10 . The non-transitory machine-readable medium of claim 7 , wherein treating the second training dataset as being poisoned comprises passing on a retraining opportunity for the AI model presented by the second training dataset.

11 . The non-transitory machine-readable medium of claim 10 , wherein passing on the retraining opportunity comprises discarding the second training dataset.

12 . The non-transitory machine-readable medium of claim 7 , wherein the causal relationship is identified using a causal model comprising nodes and edges between the nodes, the nodes correspond to portions of the first training data set, and the edges between the nodes correspond to relationships between the portions of the first training data set corresponding to the nodes.

13 . The non-transitory machine-readable medium of claim 7 , wherein making the identification comprises, at least:
determining that a predetermined amount of new training data for retraining the first instance of the AI model has been collected from data sources; and using the predetermined amount of the new training data as the second training dataset.

14 . A data processing system, comprising:
a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for identifying poisoned training data used for training an artificial intelligence (AI) model, the operations comprising:
making an identification that a second training dataset usable to retrain a first instance of the AI model is available, and
determining, based on the identification and before using the second training dataset to retrain the first instance of the AI model, whether the second t

Assignees

• Dell Products Lp

Inventors

• Ezrielev Ofir
• Savir Amihai
• Kushnir Tomer

Classifications

• G06F21/64 Primary

  Protecting data integrity, e.g. using checksums, certificates or signatures · CPC title