Explanatory dropout for machine learning models

Patent metadata

Field	Value
Publication number	US-12585996-B2
Application number	US-202217972510-A
Country	US
Kind code	B2
Filing date	Oct 24, 2022
Priority date	Oct 24, 2022
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Explanatory dropout systems and methods for improving a computer implemented machine learning model are provided using on-manifold/on-distribution evaluation of dropout of key features to explain model outputs. The machine learning model is trained using a plurality of input examples, including input records with explicit dropout operators applied effectuating the removal of influence of features associated with an explanation reason class. One or more dropout operators may be stochastically applied to one or more input examples. The procedure includes on-manifold/on-distribution evaluation of the machine learning model under conditions of absence or presence of the one or more dropout operators for reliable calculation of numerical statistics associated with reason classes to yield model explanations. The training and evaluation procedures present advantages over traditional off-manifold or off-distribution perturbative explanation procedures.

First claim

What is claimed is:

1 . An explanatory dropout method stored as instructions in at least one non-transitory machine-readable medium, wherein the instructions are executed by one or more processors for improving a computer implemented machine learning model, the execution of the instructions by the one or more processors causing a computing machine to perform operations including:
processing a machine learning model, the machine learning model trained using a plurality of input examples provided to the machine learning model, by stochastically applying one or more dropout operators to one or more input examples, the training being based on conditions including an absence or a presence of the one or more dropout operators applied to the one or more input examples; evaluating the machine learning model under the conditions of absence and presence of the one or more dropout operators applied to the one or more input examples; computing explanatory statistics for the machine learning model based on the machine learning model's output generated under the conditions of absence and presence of one or more dropout operators applied to the one or more input examples, and generating one or more reason codes associated with the one or more dropout operators, wherein the one or more reason codes provide an explanation for how the machine learning model behaves when the one or more dropout operators are present or absent.

2 . The method of claim 1 , wherein at least one of the one or more dropout operators is associated with a set of inputs corresponding to a reason group, the set of inputs having similar explanation reasons.

3 . The method of claim 1 , wherein the machine learning model is evaluated at scoring time on a first input vector to generate a score.

4 . The method of claim 3 , wherein the one or more reason codes are generated along with the score.

5 . The method of claim 4 , wherein at least a first reason code from among the one or more reason codes corresponds to at least a first reason group.

6 . The method of claim 5 , wherein the first reason group is associated with a computation using the machine learning model's output observed in various dropout and non-dropout conditions.

7 . The method of claim 5 , wherein the first reason group is associated with a most influential explanatory statistic computed for the machine learning model based on the differences in the machine learning model's output between the dropout conditions, wherein the machine learning model has been trained on both dropout and non-dropout conditions.

8 . The method of claim 1 , wherein the one or more dropout operators perform an elementwise multiplicative function upon a first input vector.

9 . The method of claim 2 , wherein the one or more dropout operators are defined per reason group, an operator denoted by D k [⋅], with k ranging over a set of reason groups, and R k denoting a set of input indices corresponding to reason group k:

D
k

[

x
j

]

=

{

α
k

·

x
j

if
⁢

j

∉

R
k

0

if
⁢

j

∈

R
k

}

10 . The method of claim 9 , wherein value of ax is one of:
α k =1, α k =D/(D−|R k |), with D overall number of input features and |R k | being the number of input features associated with a reason group k,

α
k

=

❘
"\[LeftBracketingBar]"

R

❘
"\[RightBracketingBar]"

❘
"\[LeftBracketingBar]"

R

❘
"\[RightBracketingBar]"

-
1

for all α k , with |R| being the total number of reason groups, or
α k is a free parameter to be optimized in the machine learning training process.

11 . The method of claim 1 , wherein the one or more dropout operators are chosen stochastically per example during training under a distribution according to at least one or more of the following approaches: choosing a certain fraction of examples to be subject to dropout, for a chosen example, choosing one dropout operator at random, from a distribution which may or may not be uniform over one or more reason groups; choosing whether to apply at least one of the dropout operators independently according to a binary probabilistic choice per operator, which may or may not be uniform across the reason groups, wherein probabilistic parameters are chosen to ensure that there is a reasonable probability that no dropout operator will be applied to the chosen example; choosing whether to apply no dropout or at least one of the dropout operators according to a procedure compatible with a stepwise explanatory procedure or a K-tuple explanatory procedure, choosing one or more distinct dropout operators to be applied to be a randomly selected value between zero and up to number of steps in the stepwise explanatory procedure or a cardinality of a K-tuple, and choosing identities of the one or more distinct dropout operators at random, from a distribution which may or may not be random over the reason groups; or choosing a set of dropout operators to apply or not to apply according to a Shapley sampling distribution.

12 . The method of claim 3 , wherein a unary explanatory dropout statistic is given by ι k (x)=M[x]−M[D k [x]], with M[x] denoting a machine learning model score upon input vector x, and D k [x] denoting a dropout operator dropping out inputs with a reason group k, and wherein the machine learning model has been trained on both dropout

Assignees

• Fair Isaac Corp

Inventors

• Kennel Matthew
• Zoldi Scott

Classifications

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06N3/082 Primary

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title