System and method for machine learning fairness testing

Patent metadata

Field	Value
Publication number	US-2022114399-A1
Application number	US-202117497755-A
Country	US
Kind code	A1
Filing date	Oct 8, 2021
Priority date	Oct 8, 2020
Publication date	Apr 14, 2022
Grant date	—

Abstract

Systems and methods for diagnosing and testing fairness of machine learning models based on detecting individual violations of group definitions of fairness, via adversarial attacks that aim to perturb model inputs to generate individual violations. The systems and methods employ auxiliary machine learning models using a local surrogate for identifying group membership and assess fairness by measuring the transferability of attacks from this model. The systems and methods generate fairness indicator values indicative of discrimination risk due to the target predictions generated by the machine learning model, by comparing gradients of the machine learning model to gradients of an auxiliary machine learning model.

First claim

What is claimed is:

1 . A system for estimating fairness of a machine learning model, the system comprising:
one or more processors operating in conjunction with computer memory, the one or more processors configured to:
receive data representative of a value of an input variable of the machine learning model, the machine learning model configured to generate target predictions based on the input variable;
generate a first vector indicative of a gradient of the machine learning model evaluated at the value of the input variable;
generate a second vector using an auxiliary machine learning model configured to generate predictions indicative of one or more protected attributes based on the input variable, the second vector indicative of a gradient of the auxiliary machine learning model evaluated at the value of the input variable;
compare the first vector to the second vector to generate a fairness indicator value; and
generate output data representative of the fairness indicator value;
wherein the fairness indicator value is indicative of discrimination risk in the target predictions generated by the machine learning model.

2 . The system of claim 1 , wherein comparing the first vector to the second vector includes using a projection of the first vector on to the second vector.

3 . The system of claim 1 , wherein comparing the first vector to the second vector includes using a norm of a projection value, the projection value obtained by projecting the first vector on to the second vector and dividing by a L2 norm of the second vector.

4 . The system of claim 1 , wherein the machine learning model is a first supervised learning model, and the auxiliary machine learning model is a second supervised learning model trained at least partially based on known values of the one or more protected attributes.

5 . The system of claim 1 , wherein the second vector is representative of an output of a sign function of the gradient of the auxiliary machine learning model evaluated at the value of the input variable.

6 . The system of claim 1 , wherein the second vector is indicative of a modified gradient when the gradient of the auxiliary machine learning model is associated with out-of-distribution predictions of the auxiliary machine learning model.

7 . The system of claim 1 , wherein a plurality of fairness indicator values are generated, each corresponding to different protected attributes of the one or more attributes, and the output data is indicative of whether an aggregated measure of the plurality of fairness indicators exceeds a predefined fairness threshold.

8 . The system of claim 7 , wherein each of the plurality of fairness indicator values is indicative of a covariance between the gradient of the machine learning model and the gradient of the auxiliary machine learning model.

9 . The system of claim 7 , wherein the aggregated measure is an L-p norm.

10 . The system of claim 1 , wherein the input variable includes an observable attribute correlated with at least one of the one or more protected attributes via an unobserved latent variable.

11 . A method for estimating fairness of a machine learning model, the method comprising:
receiving data representative of a value of an input variable of the machine learning model, the machine learning model configured to generate target predictions based on the input variable; generating a first vector indicative of a gradient of the machine learning model evaluated at the value of the input variable; generating a second vector using an auxiliary machine learning model configured to generate predictions indicative of one or more protected attributes based on the input variable, the second vector indicative of a gradient of the auxiliary machine learning model evaluated at the value of the input variable; comparing the first vector to the second vector to generate a fairness indicator value; and generating output data representative of the fairness indicator value,
wherein the fairness indicator value is indicative of discrimination risk in the target predictions generated by the machine learning model.

12 . The method of claim 11 , wherein comparing the first vector to the second vector includes using a projection of the first vector on to the second vector.

13 . The method of claim 11 , wherein comparing the first vector to the second vector includes using a norm of a projection value, the projection value obtained by projecting the first vector on to the second vector and dividing by a L2 norm of the second vector.

14 . The method of claim 11 , wherein the machine learning model is a first supervised learning model, and the auxiliary machine learning model is a second supervised learning model trained at least partially based on known values of the one or more protected attributes.

15 . The method of claim 11 , wherein the second vector is representative of an output of a sign function of the gradient of the auxiliary machine learning model evaluated at the value of the input variable.

16 . The method of claim 11 , wherein the second vector is indicative of a modified gradient when the gradient of the auxiliary machine learning model is associated with out-of-distribution predictions of the auxiliary machine learning model.

17 . The method of claim 11 , wherein a plurality of fairness indicator values are generated, each corresponding to different protected attributes of the one or more attributes, and the output data is indicative of whether an aggregated measure of the plurality of fairness indicators exceeds a predefined fairness threshold.

18 . The method of claim 17 , wherein each of the plurality of fairness indicator values is indicative of a covariance between the gradient of the machine learning model and the gradient of the auxiliary machine learning model.

19 . The method of claim 17 , wherein the aggregated measure is an L-p norm.

20 . A non-transitory computer readable medium storing machine interpretable instructions, which when executed by a processor, cause the processor to perform a method for estimating fairness of a machine learning model, the method comprising:
receiving data representative of a value of an input variable of the machine learning model, the machine learning model configured to generate target predictions based on the input variable; generating a first vector indicative of a gradient of the machine learning model evaluated at the value of the input variable; generating a second vector using an auxiliary machine learning model configured to generate predictions indicative of one or more protected attributes based on the input variable, the second vector indicative of a gradient of the auxiliary machine learning model evaluated at the value of the input variable; comparing the first vector to the second vector to generate a fairness indicator value; and generating output data representative of the fairness indicator value,
wherein the fairness indicator value is indicative of discrimination risk due to the target predictions generated by the machine learning model.

Assignees

• Royal Bank Of Canada

Inventors

• Castiglione Giuseppe Marcello Antonio
• Prince Simon Jeremy Damion
• Srinivasa Christopher Côté

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06F18/217 Primary

  Validation; Performance evaluation; Active pattern learning techniques · CPC title