Detecting and reducing bias in machine learning models

Patent metadata

Field	Value
Publication number	US-2020151555-A1
Application number	US-201816185715-A
Country	US
Kind code	A1
Filing date	Nov 9, 2018
Priority date	Nov 9, 2018
Publication date	May 14, 2020
Grant date	—

Abstract

A method identifies and removes bias from a machine learning model. A user/computer inputs a plurality of input training data into a machine learning system to generate an output of labeled output data. The user/computer evaluates the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment. The user/computer selects each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a labeled output data subset, and then creates additional labeling for the labeled output data subset. The user/computer utilizes the additional labeling to distinguish each labeled training data from labeled output data subset as being mislabeled and biased, and then adjusts the learning machine based on the labeled output data subset being mislabeled and biased.

First claim

What is claimed is:

1 . A method comprising:
inputting a plurality of input training data into a machine learning model in a machine learning system to generate an output of labeled output data; evaluating the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment; selecting each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a mislabeled output data subset; and adjusting the machine learning system based on the mislabeled output data subset.

2 . The method of claim 1 , wherein the mislabeled output data subset is a result of bias by human labelers for the plurality of input training data.

3 . The method of claim 1 , further comprising:
creating additional labeling for the mislabeled output data subset to create additional labeling for the mislabeled output data subset; and utilizing the additional labeling to distinguish each labeled training data from the mislabeled output data subset as being properly labeled.

4 . The method of claim 1 , further comprising:
adjusting the machine learning model by collecting and inputting additional representative training data into the machine learning system.

5 . The method of claim 1 , wherein the consistency metric is an inter-annotator agreement (IAA) between annotators that evaluate the labeled output data.

6 . The method of claim 1 , wherein the machine learning system is a traditional neural network, and wherein the plurality of input training data is generated from a data document.

7 . The method of claim 1 , wherein the machine learning system is a convolutional neural network, and wherein the plurality of input training data is generated from a photograph.

8 . A computer program product comprising a computer readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, and wherein the program code is readable and executable by a processor to perform a method comprising:
inputting a plurality of input training data into a machine learning model in a machine learning system to generate an output of labeled output data; evaluating the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment; selecting each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a mislabeled output data subset; and adjusting the machine learning system based on the mislabeled output data subset.

9 . The computer program product of claim 8 , wherein the mislabeled output data subset is a result of bias by human labelers for the plurality of input training data.

10 . The computer program product of claim 8 , wherein the method further comprises:
creating additional labeling for the mislabeled output data subset to create additional labeling for the mislabeled output data subset; and utilizing the additional labeling to distinguish each labeled training data from the mislabeled output data subset as being properly labeled.

11 . The computer program product of claim 8 , wherein the method further comprises:
adjusting the machine learning model by collecting and inputting additional representative training data into the machine learning system.

12 . The computer program product of claim 8 , wherein the consistency metric is an inter-annotator agreement (IAA) between annotators that evaluate the labeled output data.

13 . The computer program product of claim 8 , wherein the machine learning system is a traditional neural network, and wherein the plurality of input training data is generated from a data document.

14 . The computer program product of claim 8 , wherein the machine learning system is a convolutional neural network, and wherein the plurality of input training data is generated from a photograph.

15 . The computer program product of claim 8 , wherein the program code is provided as a service in a cloud environment.

16 . A computer system comprising one or more processors, one or more computer readable memories, and one or more computer readable non-transitory storage mediums, and program instructions stored on at least one of the one or more computer readable non-transitory storage mediums for execution by at least one of the one or more processors via at least one of the one or more computer readable memories, the stored program instructions executed to perform a method comprising:
inputting a plurality of input training data into a machine learning model in a machine learning system to generate an output of labeled output data; evaluating the labeled output data according to a consistency metric to associate the labeled output data with a corresponding consistency assessment; selecting each labeled output data having a consistency assessment indicating a consistency assessment that is greater than a predetermined threshold to form a mislabeled output data subset; and adjusting the machine learning system based on the mislabeled output data subset.

17 . The computer system of claim 16 , wherein the method further comprises:
creating additional labeling for the mislabeled output data subset to create additional labeling for the mislabeled output data subset; and utilizing the additional labeling to distinguish each labeled training data from the mislabeled output data subset as being properly labeled.

18 . The computer system of claim 16 , wherein the method further comprises:
adjusting the machine learning model by collecting and inputting additional representative training data into the machine learning system.

19 . The computer system of claim 16 , wherein the consistency metric is an inter-annotator agreement (IAA) between annotators that evaluate the labeled output data.

20 . The computer system of claim 16 , wherein the stored program instructions are provided as a service in a cloud environment.

Assignees

• IBM

Inventors

• Kozhaya Joseph
• Kwatra Shikhar
• Allen Corville O
• Freed Andrew R

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/092 Primary

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title