Determining data representative of bias within a model

Patent metadata

Field	Value
Publication number	US-11636386-B2
Application number	US-201916690686-A
Country	US
Kind code	B2
Filing date	Nov 21, 2019
Priority date	Nov 21, 2019
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

Methods, systems, and computer program products for determining data representative of bias within a model are provided herein. A computer-implemented method includes obtaining a first dataset on which a model was trained, wherein the first dataset contains protected attributes, and a second dataset on which the model was trained, wherein the protected attributes have been removed from the second dataset; identifying, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith; determining bias among at least a portion of the identified correlated attributes; and outputting, to at least one user, identifying information pertaining to the one or more instances of bias.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining, as input, (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
identifying, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith;
determining bias among at least a portion of the identified correlated attributes, wherein said determining comprises:
mapping at least two classes of data points associated with the correlated attributes in the second dataset to a set of values associated with the one or more protected attributes in the first dataset; and
identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;

outputting, to at least one user, identifying information pertaining to the one or more instances of bias; and
performing at least one validity analysis on the one or more identified instances of bias, wherein said performing the at least one validity analysis comprises:
training, using at least one training dataset related to the one or more identified instances of bias, and implementing one or more of a nearest neighbors classifier and a nearest neighbors regressor, wherein implementing the one or more of a nearest neighbors classifier and a nearest neighbors regressor comprises identifying, using the one or more of a nearest neighbors classifier and a nearest neighbors regressor, one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings with one or more modified class designations associated with the one or more instances of bias; and
calculating at least one weighted distance between (i) the one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings and (ii) the at least a portion of the data points in the mappings, wherein each weight represents a normalized correlation coefficient between at least one of the correlated attributes and at least one of the one or more protected attributes;

wherein the method is carried out by at least one computing device.

2. The computer-implemented method of claim 1 , wherein said identifying the one or more correlated attributes in the second dataset comprises cross-validating at least one linear model in connection with at least a portion of the attributes in the second data set and the one or more protected attributes in the first dataset.

3. The computer-implemented method of claim 2 , wherein the at least one linear model comprises a logistic regression.

4. The computer-implemented method of claim 2 , wherein the at least one linear model comprises a linear regression.

5. The computer-implemented method of claim 1 , wherein said identifying the one or more correlated attributes in the second dataset comprises filtering normalized weights corresponding to attributes in the first dataset which are greater than a predetermined threshold.

6. The computer-implemented method of claim 1 , wherein the at least two classes of data points comprise a majority class and a minority class, distinguished in accordance with a predetermined threshold value.

7. The computer-implemented method of claim 1 , wherein the one or more instances of bias are represented by one or more data points in at least one of the first dataset and the second dataset.

8. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:
obtain, as input, (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
identify, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith;
determine bias among at least a portion of the identified correlated attributes, wherein said determining comprises:
mapping at least two classes of data points associated with the correlated attributes in the second dataset to a set of values associated with the one or more protected attributes in the first dataset; and
identifying one or more instances of bias by observing a change to one or more of the values in the mappings in response to modifying one or more class designations among the data points in the mappings;

output, to at least one user, identifying information pertaining to the one or more instances of bias; and
perform at least one validity analysis on the one or more identified instances of bias, wherein said performing the at least one validity analysis comprises:
training, using at least one training dataset related to the one or more identified instances of bias, and implementing one or more of a nearest neighbors classifier and a nearest neighbors regressor, wherein implementing the one or more of a nearest neighbors classifier and a nearest neighbors regressor comprises identifying, using the one or more of a nearest neighbors classifier and a nearest neighbors regressor, one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings with one or more modified class designations associated with the one or more instances of bias; and
calculating at least one weighted distance between (i) the one or more data points from the at least one training dataset which are within a predetermined proximity to at least a portion of the data points in the mappings and (ii) the at least a portion of the data points in the mappings, wherein each weight represents a normalized correlation coefficient between at least one of the correlated attributes and at least one of the one or more protected attributes.

9. The computer program product of claim 8 , wherein said identifying the one or more correlated attributes in the second dataset comprises cross-validating at least one linear model in connection with at least a portion of the attributes in the second data set and the one or more protected attributes in the first dataset.

10. The computer program product of claim 9 , wherein the at least one linear model comprises one or more of a logistic regression and a linear regression.

11. The computer program product of claim 8 , wherein said identifying the one or more correlated attributes in the second dataset comprises filtering normalized weights corresponding to attributes in the first dataset which are greater than a predetermined threshold.

12. A system comprising:
a memory; and
at least one processor operably coupled to the memory and configured for:
obtaining, as input, (i) a first dataset on which a model was trained, wherein the first dataset contains one or more protected attributes, and (ii) a second dataset on which the model was trained, wherein the one or more protected attributes have been removed from the second dataset;
identifying, for each of the one or more protected attributes in the first dataset, one or more attributes in the second dataset correlated therewith;
determining bias among at least a portion o

Assignees

• IBM

Inventors

• Lohia Pranay Kumar
• Saha Diptikalyan
• Bhide Manish Anand
• Mehta Sameep

Classifications

• G06N7/00

  Computing arrangements based on specific mathematical models · CPC title

• G06F18/217 Primary

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F21/6254

  by anonymising data, e.g. decorrelating personal data from the owner's identification · CPC title

• G06N20/00 Primary

  Machine learning · CPC title