Framework for explainability with recourse of black-box trained classifiers and assessment of fairness and robustness of black-box trained classifiers

Patent metadata

Field	Value
Publication number	US-11645620-B2
Application number	US-201916657397-A
Country	US
Kind code	B2
Filing date	Oct 18, 2019
Priority date	Mar 15, 2019
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A method, system and computer-readable storage medium for performing a counterfactual generation operation. The counterfactual generation operation includes: receiving a subject data point; classifying the data point via a trained classifier, the classifying providing a classified data point; identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model; and, providing the counterfactual to a destination.

First claim

What is claimed is:

1. A computer-implementable method for performing a counterfactual generation operation, the method comprising:
receiving a subject data point;
classifying the data point via a trained classifier, the classifying providing a classified data point;
identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model; and,
performing an augmented intelligence system explainability with recourse operation using the counterfactual, the augmented intelligence system explainability with recourse operation being used when performing an augmented intelligence system assurance operation, the augmented intelligence system explainability with recourse operation using the counterfactual to employ as a recourse to changing a particular decision made by the model, the augmented intelligence system assurance operation ensuring augmented intelligence performance of a cognitive computing operation performed by an augmented intelligence system, the ensuring augmented intelligence performance ensuring one or more of transparency, interpretability, impartiality, accountability and trustworthiness of the cognitive computing operation.

2. The method of claim 1 , wherein:
the identifying comprises searching a plurality of datapoints to identify the datapoint closest to the subject datapoint.

3. The method of claim 1 , wherein:
the identifying uses a genetic algorithm.

4. The method of claim 3 , wherein:
the genetic algorithm comprises an evolutionary algorithm.

5. The method of claim 3 , wherein:
the genetic algorithm generates a set of random data points, the trained classifier providing classification for the set of random data points, a classification for at least some of the random data points differing from the classification of the subject data point.

6. The method of claim 5 , further comprising:
iteratively revising the set of random data points to provide revised data points, the revised data points being closer to the subject data point than the set of random data points while maintaining classifications that differ from the classification of the subject data point.

7. A system comprising:
a processor;
a data bus coupled to the processor; and
a non-transitory, computer-readable storage medium embodying computer program code, the non-transitory, computer-readable storage medium being coupled to the data bus, the computer program code interacting with a plurality of computer operations and comprising instructions executable by the processor and configured for:
receiving a subject data point;
classifying the data point via a trained classifier, the classifying providing a classified data point;
identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model; and,
performing an augmented intelligence system explainability with recourse operation using the counterfactual, the augmented intelligence system explainability with recourse operation being used when performing an augmented intelligence system assurance operation, the augmented intelligence system explainability with recourse operation using the counterfactual to employ as a recourse to changing a particular decision made by the model, the augmented intelligence system assurance operation ensuring augmented intelligence performance of a cognitive computing operation performed by an augmented intelligence system, the ensuring augmented intelligence performance ensuring one or more of transparency, interpretability, impartiality, accountability and trustworthiness of the cognitive computing operation.

8. The system of claim 7 , wherein:
the identifying comprises searching a plurality of datapoints to identify the datapoint closest to the subject datapoint.

9. The system of claim 7 , wherein:
the identifying uses a genetic algorithm.

10. The system of claim 9 , wherein:
the genetic algorithm comprises an evolutionary algorithm.

11. The system of claim 9 , wherein:
the genetic algorithm generates a set of random data points, the trained classifier providing classification for the set of random data points, a classification for at least some of the random data points differing from the classification of the subject data point.

12. The system of claim 11 , wherein the instructions executable by the processor are further configured for:
iteratively revising the set of random data points to provide revised data points, the revised data points being closer to the subject data point than the set of random data points while maintaining classifications that differ from the classification of the subject data point.

13. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
receiving a subject data point;
classifying the data point via a trained classifier, the classifying providing a classified data point;
identifying a counterfactual using the classified data point, the counterfactual comprising another datapoint, the another data point being close to the subject data point, the another data point resulting in production of a different outcome when provided to a model when compared to an outcome resulting from the subject data point being provided to the model; and,
performing augmented intelligence system explainability with recourse operation using the counterfactual, the augmented intelligence system explainability with recourse operation being used when performing an augmented intelligence system assurance operation, the augmented intelligence system explainability with recourse operation using the counterfactual to employ as a recourse to changing a particular decision made by the model, the augmented intelligence system assurance operation ensuring augmented intelligence performance of a cognitive computing operation performed by an augmented intelligence system, the ensuring augmented intelligence performance ensuring one or more of transparency, interpretability, impartiality, accountability and trustworthiness of the cognitive computing operation.

14. The non-transitory, computer-readable storage medium of claim 13 , wherein:
the identifying comprises searching a plurality of datapoints to identify the datapoint closest to the subject datapoint.

15. The non-transitory, computer-readable storage medium of claim 13 , wherein:
the identifying uses a genetic algorithm.

16. The non-transitory, computer-readable storage medium of claim 15 , wherein:
the genetic algorithm comprises an evolutionary algorithm.

17. The non-transitory, computer-readable storage medium of claim 15 , wherein:
the genetic algorithm generates a set of random data points, the trained classifier providing classification for the set of random data points, a classification for at least some of the random data points differing from the classification of the subject data point.

18. The non-transitory, computer-readable storage medium of claim 13 , wherein the computer executable instructions are fur

Assignees

• Cognitive Scale Inc
• Tecnotree Tech Inc

Inventors

• Ghosh Joydeep
• Sharma Shubham
• Henderson Jessica
• Sanchez Matthew

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06N99/007

  Molecular computers, i.e. using inorganic molecules (using biomolecules G06N3/002) · CPC title

• G06Q10/10 Primary

  Office automation; Time management · CPC title

• G06N3/126

  Evolutionary algorithms, e.g. genetic algorithms or genetic programming · CPC title

• G06F18/2111

  by using evolutionary computational techniques, e.g. genetic algorithms · CPC title