Bias detection and reduction in machine-learning techniques

Patent metadata

Field	Value
Publication number	US-12596957-B2
Application number	US-202218046661-A
Country	US
Kind code	B2
Filing date	Oct 14, 2022
Priority date	Oct 15, 2021
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

In some aspects, a computing system can improve a machine learning model for risk assessment by removing or reducing bias in the machine learning model. The training process for the machine learning model can include training the machine learning model using training samples, obtaining data for a protected attribute, and calculating a bias metric using the data for the protected attribute and data obtained from the trained machine learning model. Based on the bias metric, bias associated with the machine learning model can be detected. The machine learning model can be modified based on the detected bias and re-trained. The re-trained machine learning model can be used to predict a risk indicator for a target entity. The predicted risk indicator can be transmitted to a remote computing device and be used for controlling access of the target entity to one or more interactive computing environments.

First claim

The invention claimed is:

1 . A method that includes one or more processing devices performing operations comprising:
determining, using a machine learning model trained using a training process, a risk indicator for a target entity from predictor variables associated with the target entity, wherein the risk indicator indicates a level of risk associated with the target entity, wherein the training process includes operations comprising:
training the machine learning model using training samples comprising training predictor variables and training outputs corresponding to the training predictor variables,
obtaining a first data set for a first subgroup defining a protected attribute;
obtaining a second data set for a second subgroup different from the first subgroup;
calculating a bias metric using the first data set, the second data set, and predicted data obtained from the trained machine learning model, wherein the bias metric comprises a calibrated log-odds difference representing a difference between a first comparison of the first data set to the predicted data and a second comparison of the second data set to the predicted data;
determining that a bias is detected based on the bias metric;
modifying the machine learning model based on the detected bias;
re-training the machine learning model; and
transmitting, to a remote computing device, a responsive message including at least the risk indicator for use in controlling access of the target entity to one or more interactive computing environments.

2 . The method of claim 1 , wherein the protected attribute is one of an individual level protected attribute or a geographic level protected attribute and obtaining data for the protected attribute comprises estimating the data for the geographic level protected attribute based on census released data and mapping to individuals.

3 . The method of claim 1 , wherein the second subgroup complements the first subgroup such that combining the first subgroup with the second subgroup represents a full group representative of the protected attribute.

4 . The method of claim 3 , wherein determining that a bias is detected based on the bias metric comprises determining that an absolute value of the bias metric is higher than a threshold value for the calibrated log-odds difference.

5 . The method of claim 1 , wherein the bias metric comprises a correlation metric that comprises a first correlation between values of a training predictor variable and the data of the protected attribute and a second correlation between outputs of the machine learning model and the data of the protected attribute.

6 . The method of claim 5 , wherein determining that a bias is detected based on the bias metric comprises determining that at least one of the first correlation and the second correlation is higher than a threshold value for the correlation.

7 . The method of claim 1 , wherein modifying the machine learning model based on the detected bias comprises one or more of:
removing a predictor variable for which the bias metric indicates a bias; re-defining a predictor variable for which the bias metric indicates a bias; or modifying the training samples based on the detected bias.

8 . A system comprising:
a processing device; and a memory device in which instructions executable by the processing device are stored for causing the processing device to perform operations comprising:
determining, using a machine learning model trained using a training process, a risk indicator for a target entity from predictor variables associated with the target entity, wherein the risk indicator indicates a level of risk associated with the target entity, wherein the training process includes operations comprising:
training the machine learning model using training samples comprising training predictor variables and training outputs corresponding to the training predictor variables,
obtaining a first data set for a first subgroup defined by a protected attribute;
obtaining a second data set for a second subgroup different from the first subgroup;
calculating a bias metric using the first data set and the second data set and predicted data obtained from the trained machine learning model, wherein the bias metric comprises a calibrated log-odds difference representing a difference between a first comparison of the first data set compared to the predicted data and a second comparison of the second data set compared to the predicted data;
determining that a bias is detected based on the bias metric;
modifying the machine learning model based on the detected bias;
re-training the machine learning model; and

transmitting, to a remote computing device, a responsive message including at least the risk indicator for use in controlling access of the target entity to one or more interactive computing environments.

9 . The system of claim 8 , wherein the protected attribute is one of an individual level protected attribute or a geographic level protected attribute and obtaining data for the protected attribute comprises estimating the data for the geographic level protected attribute based on census released data and mapping to individuals.

10 . The system of claim 8 , wherein the second subgroup complements the first subgroup such that combining the first subgroup with the second subgroup represents a full group representative of the protected attribute.

11 . The system of claim 10 , wherein the operation of determining that a bias is detected based on the bias metric comprises determining that an absolute value of the bias metric is higher than a threshold value for the calibrated log-odds difference.

12 . The system of claim 8 , wherein the bias metric comprises a correlation metric that comprises a first correlation between values of a training predictor variable and the data of the protected attribute and a second correlation between outputs of the machine learning model and the data of the protected attribute.

13 . The system of claim 12 , wherein the operation of determining that a bias is detected based on the bias metric comprises determining that at least one of the first correlation and the second correlation is higher than a threshold value for the correlation.

14 . The system of claim 8 , wherein the operation of modifying the machine learning model based on the detected bias comprises one or more of:
removing a predictor variable for which the bias metric indicates a bias; re-defining a predictor variable for which the bias metric indicates a bias; or modifying the training samples based on the detected bias.

15 . A non-transitory computer-readable storage medium having program code that is executable by a processor device to cause a computing device to perform operations, the operations comprising:
determining, using a machine learning model trained using a training process, a risk indicator for a target entity from predictor variables associated with the target entity, wherein the risk indicator indicates a level of risk associated with the target entity, wherein the training process includes operations comprising:
training the machine learning model using training samples comprising training predictor variables and training outputs corresponding to the training predictor variables,
obtaining a first data set for a first subgroup defined by a protected attribute;
obtaining a second data set for a second subgroup different from the first subgroup;
calculating a bias metric using the first data set and the second data set and predicted data obtained from the trained machine learning model, wher

Assignees

• Equifax Inc

Inventors

• Zou Mufeng
• Veeravelly Swathi
• Bruhn Marcus

Classifications

• G06N20/10

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06F18/2155

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title

• G06F40/44

  Statistical methods, e.g. probability models · CPC title