Method and system for tracing a learning source of an explainable artificial intelligence model

Patent metadata

Field	Value
Publication number	US-11775857-B2
Application number	US-201816043209-A
Country	US
Kind code	B2
Filing date	Jul 24, 2018
Priority date	Jun 5, 2018
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

This disclosure relates generally to artificial intelligence system, and more particularly to method and system for tracing a learning source of an explainable artificial intelligence (AI) model. In one example, the method may include receiving a desired behavior of the explainable AI model with respect to input data, generating a learning graph based on similarities among a plurality of learning sources with respect to the input data for the desired behavior and for a current behavior, retracing a learning of the explainable AI model by iteratively comparing the learning graph for the desired behavior and for the current behavior at each of a plurality of layers of the explainable AI model starting from an outer layer, and detecting the learning source responsible for the current behavior based on the retracing.

First claim

What is claimed is:

1. A method of tracing a learning source of an explainable artificial intelligence (AI) model, the method comprising:
receiving, by a tracing device, a desired behavior of the explainable AI model with respect to input data, wherein the desired behavior comprises a correct classification of the input data;
determining, by the tracing device, similarities among a plurality of learning sources with respect to the input data for the desired behavior and for a current behavior, wherein the similarities are determined using a set of filters used in a first layer of convolutional neural network (CNN);
generating, by the tracing device, a learning graph based on the similarities among the plurality of learning sources with respect to the input data for the desired behavior and for a current behavior, wherein the current behavior comprises an erroneous classification of the input data;
retracing, by the tracing device, a learning of the explainable AI model by iteratively comparing the learning graph for the desired behavior and for the current behavior at each of a plurality of layers of the explainable AI model starting from an outer layer, wherein the learning graph is based on one or more probabilities generated for a layer through Inverse Bayesian Fusion (IBF) so as to separate learning components of one or more of a randomly selected set of learning sources;
detecting, by the tracing device, the learning source from the randomly selected set of learning sources responsible for the current behavior based on the retracing, wherein the learning source is detected based on selecting the learning source for an output with a least distance metric in a direction of reducing gradient, and wherein the retracing is repeated until the learning source responsible for the current behavior is detected;
correcting, by the tracing device, a classification model by unlearning with respect to the learning source responsible for the erroneous classification and learning again with respect to a corrected learning source;
providing, by the tracing device, feedback about performance of the corrected learning source in order to check consistency of the corrected learning source;
providing, by a display module of the tracing device, an interface for displaying the feedback about the performance to a user; and
receiving, by the tracing device, an input from the user related to invoking another iteration of execution based on the feedback, thereby increasing a degree of confidence.

2. The method of claim 1 , wherein the learning sources comprises at least one of a cluster of training data, a training environment, or an object-class pair applied by the user.

3. The method of claim 1 , further comprising generating a sequence graph by organizing the plurality of learning sources in a hierarchical manner.

4. The method of claim 3 , wherein generating the learning graph comprises generating the learning graph with the randomly selected set of learning sources from among the plurality of learning sources based on the sequence graph.

5. The method of claim 4 , wherein the randomly selected set of learning sources comprises a randomly selected learning source, a learning source hierarchically above the randomly selected learning source, and a learning source hierarchically below the randomly selected learning source.

6. The method of claim 1 , wherein the one or more probabilities for the layer is computed based on one or more distance metrics between output of the layer and outputs of a previous layer.

7. The method of claim 6 , wherein each of the one or more distance metrics is a function of one or more distances between an output of the layer and outputs of the previous layer and one or more probabilities of the previous layer.

8. The method of claim 1 , further comprising validating the updated AI model using additional test data.

9. A system for tracing a learning source of an explainable artificial intelligence (AI) system, the system comprising:
a tracing device comprising at least one processor and a computer-readable medium storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
receiving a desired behavior of the explainable AI model with respect to input data, wherein the desired behavior a correct classification of the input data;
determining similarities among a plurality of learning sources with respect to the input data for the desired behavior and for a current behavior, wherein the similarities are determined using a set of filters used in a first layer of convolutional neural network (CNN);
generating a learning graph based on the similarities among the plurality of learning sources with respect to the input data for the desired behavior and for a current behavior, wherein the current behavior comprises an erroneous classification of the input data;
retracing a learning of the explainable AI model by iteratively comparing the learning graph for the desired behavior and for the current behavior at each of a plurality of layers of the explainable AI model starting from an outer layer wherein the learning graph is based on one or more probabilities generated for a layer through Inverse Bayesian Fusion (IBF) so as to separate learning components of one or more of a randomly selected set of learning sources;
detecting the learning source from the randomly selected set of learning sources responsible for the current behavior based on the retracing, wherein the learning source is detected based on selecting the learning source for an output with a least distance metric in a direction of reducing gradient, and wherein the retracing is repeated until the learning source responsible for the current behavior is detected;
correcting a classification model by unlearning with respect to the learning source responsible for the erroneous classification and learning again with respect to a corrected learning source;
providing feedback about performance of the corrected learning source in order to check consistency of the corrected learning source;
providing an interface for displaying the feedback about the performance to a user; and
receiving an input from the user related to invoking another iteration of execution based on the feedback, thereby increasing a degree of confidence.

10. The system of claim 9 , wherein the operations further comprise generating a sequence graph by organizing the plurality of learning sources in a hierarchical manner.

11. The system of claim 10 , wherein generating the learning graph comprises generating the learning graph with the randomly selected set of learning sources from among the plurality of learning sources based on the sequence graph, and wherein the randomly selected set of learning sources comprises a randomly selected learning source, a learning source hierarchically above the randomly selected learning source, and a learning source hierarchically below the randomly selected learning source.

12. The system of claim 1 , wherein the one or more probabilities for the layer is computed based on one or more distance metrics between output of the layer and outputs of a previous layer, and wherein each of the one or more distance metrics is a function of one or more distances between an output of the layer and outputs of the previous layer and one or more probabilities of the previous layer.

13. The system of claim 9 , further comprising:
validating the updated AI model using additional test data.

14. A non-transitory computer-readable medium storing computer-executable instructions for:
receiving a desired behavior of

Assignees

• Wipro Ltd

Inventors

• Ramachandra Iyer Manjunath

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06F18/285

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title