Detecting representative bias within image data sets

Patent metadata

Field	Value
Publication number	US-12400305-B2
Application number	US-202117451898-A
Country	US
Kind code	B2
Filing date	Oct 22, 2021
Priority date	Oct 22, 2021
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

One example method includes determining representation bias in a data set. A bias detection engine is trained using a data set that is sufficiently diversified and/or unbiased. Once trained, test data sets can be evaluated by the bias detection engine to determine an amount of representation bias in the test data sets. The representation bias can be visually conveyed to a user and suggestions on how to reduce the representation bias may be provided and/or implemented to reduce the representation bias in the test data set. Suggestions can be implemented by adding or removing data from the test data that will reduce the representation bias.

First claim

What is claimed is:

1. A method, comprising:
training a bias detection engine with a training data set that is unbiased or substantially unbiased, wherein the training data set covers all groups of a population represented by the training data set, wherein all the groups are of equal sizes or wherein the training data set is configured such than any one group of the population is not over-represented;
receiving input to the bias detection engine, the input including a test data set, features related to the test data set, and parameters of the features;
processing the input by the bias detection engine by clustering data points in the test data set into clusters;
outputting, by the bias detection engine, a diversity score for the test data set based on the clusters, wherein the diversity score represents an amount of representation bias in the test data set; and
wherein the output includes a distribution per feature and/or parameter and a comparison of the distribution to corresponding distributions of the training data set and wherein the bias detection engine is configured to prevent over-sampling.

2. The method of claim 1 , wherein the clusters are scored using a Shannon score, further comprising determining a threshold score.

3. The method of claim 2 , wherein the test data set is biased when the diversity score is below the threshold score and wherein the test data set is sufficiently unbiased when the diversity score is greater than the threshold score.

4. The method of claim 1 , further comprising generating additional scores including at least one of:
a distribution per feature/parameter;
a comparison of distribution between the test data set and a base data set;
a comparison of the test data set to an average data set;
a comparison of the test data set to other data sets having the same features; or
an improvement in the diversity score for the test data set over time.

5. The method of claim 1 , further comprising generating an explanation of the diversity score.

6. The method of claim 4 , further comprising generating a visualization of the diversity score and/or the additional scores.

7. The method of claim 1 , further comprising generating an explanation of implications of the diversity score.

8. The method of claim 1 , further comprising generating suggestions for improving the test data set.

9. The method of claim 1 , wherein the test data set includes images of faces and wherein the bias detection engine is trained with a training data set of images of faces.

10. The method of claim 1 , wherein the training data set covers all groups of a population represented by the training data set, wherein all groups are of equal sizes or wherein the training data are configured such than any one group of the population not over-represented.

11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
training a bias detection engine with a training data set that is unbiased or substantially unbiased, wherein the training data set covers all groups of a population represented by the training data set, wherein all the groups are of equal sizes or wherein the training data set is configured such than any one group of the population is not over-represented;
receiving input to the bias detection engine, the input including a test data set, features related to the test data set, and parameters of the features;
processing the input by the bias detection engine by clustering data points in the test data set into clusters;
outputting, by the bias detection engine, a diversity score for the test data set based on the clusters, wherein the diversity score represents an amount of representation bias in the test data set; and
wherein the output includes a distribution per feature and/or parameter and a comparison of the distribution to corresponding distributions of the training data set and wherein the bias detection engine is configured to prevent over-sampling.

12. The non-transitory storage medium of claim 11 , wherein the clusters are scored using a Shannon score, further comprising determining a threshold score.

13. The non-transitory storage medium of claim 12 , wherein the test data set is biased when the diversity score is below the threshold score and wherein the test data set is sufficiently unbiased when the diversity score is greater than the threshold score.

14. The non-transitory storage medium of claim 11 , further comprising generating additional scores including at least one of:
a distribution per feature/parameter;
a comparison of distribution between the test data set and a base data set;
a comparison of the test data set to an average data set;
a comparison of the test data set to other data sets having the same features; or
an improvement in the diversity score for the test data set over time.

15. The non-transitory storage medium of claim 14 , further comprising generating a visualization of the diversity score and/or the additional scores.

16. The non-transitory storage medium of claim 11 , further comprising generating an explanation of implications of the diversity score, generating suggestions for improving the test data set, and generating an explanation of the diversity score.

17. The non-transitory storage medium of claim 11 , further wherein the test data set includes images of faces and wherein the bias detection engine is trained with a training data set of images of faces.

18. The non-transitory storage medium of claim 11 , further wherein the training data set covers all groups of a population represented by the training data set, or wherein all groups are of equal sizes or wherein the training data are configured such that any one group in the population is not over-represented.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Shafik Farida
• Beshara Shary
• Seibel Amy N
• Bruno Eric
• Hyland Megan

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title