Faithful and efficient sample-based model explanations

Patent metadata

Field	Value
Publication number	US-12423614-B2
Application number	US-202117334889-A
Country	US
Kind code	B2
Filing date	May 31, 2021
Priority date	May 31, 2021
Publication date	Sep 23, 2025
Grant date	Sep 23, 2025

Abstract

Hessian matrix-free sample-based techniques for model explanations that are faithful to the model are provided. In one aspect, a method for explaining a machine learning model {circumflex over (θ)} (e.g., for natural language processing) is provided. The method includes: training the machine learning model {circumflex over (θ)} with training data D; obtaining a decision of the machine learning model {circumflex over (θ)}; and explaining the decision of the machine learning model {circumflex over (θ)} using training examples from the training data D.

First claim

What is claimed is:

1. A method for explaining a machine learning model {circumflex over (θ)}, the method comprising:
training the machine learning model {circumflex over (θ)} with training data D;
obtaining a decision of the machine learning model {circumflex over (θ)};
computing a set of faithful variants {{circumflex over (θ)} i } of the machine learning model {circumflex over (θ)} using the training data D by:
randomly selecting batches B of the training data D;
calculating, for each batch B i a gradient g (B i |{circumflex over (θ)}); and
computing the set of faithful variants {{circumflex over (θ)} i } of the machine learning model {circumflex over (θ)} using the gradient g(B i |{circumflex over (θ)}) for each batch B i as {circumflex over (θ)} i ={circumflex over (θ)}+η i g(B i |{circumflex over (θ)}), wherein η i is an i-specific weighting parameter; and

explaining the decision of the machine learning model {circumflex over (θ)} using a set of training examples from the set of faithful variants {{circumflex over (θ)} i } such that the decision is explained by a sum of influences of the set of faithful variants {{circumflex over (θ)} i }.

2. The method of claim 1 , wherein the machine learning model {circumflex over (θ)} is used for natural language processing.

3. The method of claim 1 , wherein explaining the decision of the machine learning model {circumflex over (θ)} comprises:
explaining a test example z′ with the training examples from the set of faithful variants {{circumflex over (θ)} i }.

4. The method of claim 3 , further comprising:
calculating importance scores for the training examples from the set of faithful variants {{circumflex over (θ)} i };
ranking the importance scores to create a ranked list of the training examples; and
providing the ranked list of the training examples as an explanation of the test example z′.

5. The method of claim 4 , wherein calculating the importance scores for the training examples comprises:
calculating an importance score TracInF (z,z′) for a training example z; and
repeating the calculating for all of the training examples.

6. The method of claim 5 , wherein the importance score TracInF (z,z′) is calculated as a sum of a product of two loss L functions, one for the training example z, and another for the test example z′.

7. The method of claim 5 , further comprising:
calculating the importance score TracInF (z,z′) as:

TracInF
⁡
(

z
,

z
′

)

=

∑
i

∇

θ
^

+

δ
i

L
(

θ
^

+

δ
i

,
z

)

⁢

∇

θ
^

+

δ
i

L
(

θ
^

+

δ
i

,

z
′

)

,

wherein δ i =η i g(z i |{circumflex over (θ)}) for loss function L, wherein z i is a given training example, and wherein η i is an i-specific weighting parameter.

8. A method for explaining a machine learning model {circumflex over (θ)}, the method comprising:
training the machine learning model {circumflex over (θ)} with training data D;
obtaining a decision of the machine learning model {circumflex over (θ)} on a test example z′;
explaining the decision of the machine learning model {circumflex over (θ)} using training examples from randomly selected batches of the training data D;
calculating individual importance scores TracInF (z,z′) for the training examples of the randomly selected batches, respectively, as:

TracInF
⁡
(

z
,

z
′

)

=

∑
i

∇

θ
^

+

δ
i

L
(

θ
^

+

δ
i

,
z

)

⁢

∇

θ
^

Assignees

• IBM

Inventors

• Zhu Yada
• Zhang Wei
• Ye Guangnan
• Cui Xiaodong

Classifications

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06F40/20

  Natural language analysis (semantic analysis of natural language G06F40/30) · CPC title