Post-hoc improvement of instance-level and group-level prediction metrics

Patent metadata

Field	Value
Publication number	US-11734585-B2
Application number	US-201816214703-A
Country	US
Kind code	B2
Filing date	Dec 10, 2018
Priority date	Dec 10, 2018
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

A post-processing method, system, and computer program product for post-hoc improvement of instance-level and group-level prediction metrics, including training a bias detector that learns to detect a sample that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, applying the bias detector on a run-time sample to select a biased sample in the run-time sample having a bias greater than the predetermined individual bias threshold bias value, and suggesting a de-biased prediction for the biased sample.

First claim

What is claimed is:

1. A post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method comprising:
training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, the sample being a member of an unprivileged group, wherein, during the training:
the bias detector perturbs a protected attribute in the payload data for the unprivileged group and computes the individual bias as an individual bias score by finding a difference between a probability of a favorable outcome for the perturbed protected attribute to original data of the payload data;
flagging the unprivileged group samples that have the individual bias greater than the predetermined individual bias threshold value; and
training the bias detector to discriminate between the flagged samples and un-flagged samples;

applying, in a run-time, the bias detector on a run-time sample to select a biased sample in the run-time sample having an individual bias greater than the predetermined individual bias threshold value; and
suggesting, in the run-time, a de-biased prediction for the biased sample by perturbing the protected attribute and checking for bias after perturbation.

2. The post-processing computer-implemented method of claim 1 , wherein the applying and the suggesting operate in a post-processing and targets the biased sample with individual bias for remediation in order to change bias of the biased sample based both on individual and group fairness metrics.

3. The post-processing computer-implemented method of claim 1 , wherein the bias detector is trained by:
for each sample point in the training set, obtaining an average individual bias after multiple perturbations;
obtaining a difference in the individual bias for a favorable class;
for the unprivileged group, setting the individual bias as the average individual bias; and
sorting samples in the training set by descending order of the difference in the individual bias to the favorable class.

4. The post-processing computer-implemented method of claim 1 , wherein the suggesting the de-biased prediction is performed by:
perturbing the protected attribute in a training set;
picking a most likely prediction for the perturbed results run through the customer model; and

changing to the most likely prediction for the perturbed results for the unprivileged group member if the most likely prediction for the perturbed results belongs to the favorable class.

5. The post-processing computer-implemented method of claim 1 , wherein a detected individual bias sample among the unprivileged group predicted by the bias detector is prioritized for a correction by the suggesting, and
wherein a user decides whether to choose an original value of the bias or the suggested de-biased prediction.

6. The post-processing computer-implemented method of claim 5 , wherein the perturbations are performed across protected attributes and an aggregate outcome is determined.

7. The post-processing computer-implemented method of claim 5 , wherein an outcome among multiple classes of the bias is chosen by one of:
looking at an aggregate prediction for each class of the multiple classes after perturbations; and
finding a most likely predicted outcome after the perturbations.

8. The post-processing computer-implemented method of claim 1 , wherein, during the run-time:
the applying applies the bias detector on the unprivileged group in the run-time to compute a likelihood of the individual bias;
testing an individually biased sample by perturbing a protected attribute and checking the outcome after perturbation; and
if the outcome after the perturbation is different from an original outcome, suggesting an individually biased sample as a de-biased prediction to an arbiter, which can choose between the original value of the bias and a de-biased prediction.

9. The post-processing computer-implemented method of claim 1 , embodied in a cloud-computing environment.

10. A post-processing computer program product for post-hoc improvement of instance-level and group-level prediction metrics, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, the sample being a member of an unprivileged group, wherein, during the training:
the bias detector perturbs a protected attribute in the payload data for the unprivileged group and computes the individual bias as an individual bias score by finding a difference between a probability of a favorable outcome for the perturbed protected attribute to original data of the payload data;
flagging the unprivileged group samples that have the individual bias greater than the predetermined individual bias threshold value; and
training the bias detector to discriminate between the flagged samples and un-flagged samples;

applying, in a run-time, the bias detector on a run-time sample to select a biased sample in the run-time sample having an individual bias greater than the predetermined individual bias threshold value; and
suggesting, in the run-time, a de-biased prediction for the biased sample by perturbing the protected attribute and checking for bias after perturbation.

11. The post-processing computer program product of claim 10 , wherein the applying and the suggesting operate in a post-processing that targets the biased sample with individual bias for remediation in order to change bias of the biased sample based both on individual and group fairness metrics.

12. The post-processing computer program product of claim 10 , wherein the bias detector is trained by:
for each sample point in the training set, obtaining an average individual bias after multiple perturbations;
obtaining a difference in the individual bias for a favorable class;
for a privileged group, setting the individual bias as the original detected individual bias;
for the unprivileged group, setting the individual bias as the average individual bias; and
sorting samples in the training set by descending order of the difference in the individual bias to the favorable class.

13. The post-processing computer program product of claim 10 , wherein the suggesting the de-biased prediction is performed by:
perturbing the protected attribute in a training set;
picking a most likely prediction for the perturbed results run through the customer model; and

changing the most likely prediction for the perturbed results for the unprivileged group member to a result for a favorable class.

14. The post-processing computer program product of claim 10 , wherein, if the bias detector returns that a particular sample has bias, then that sample is de-biased.

15. The post-processing computer program product of claim 10 , wherein a highest individual bias sample among the unprivileged group predicted by the bias detector is prioritized for a correction by the suggesting, and
wherein a user decides whether to choose an original value of the bias or the suggested de-biased prediction.

16. A post-processing system for post-hoc improvement of instance-level and group-level prediction metrics, said system compris

Assignees

• IBM

Inventors

• Bhide Manish
• Lohia Pranay
• Natesan Ramamurthy Karthikeyan
• Puri Ruchir
• Saha Diptikalyan
• Varshney Kush Raj

Classifications

• G06N5/04 Primary

  Inference or reasoning models · CPC title

• G06N20/00

  Machine learning · CPC title

• G06N20/20 Primary

  Ensemble learning · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title