Neural network negative rule extraction

Patent metadata

Field	Value
Publication number	US-11907334-B2
Application number	US-202017115610-A
Country	US
Kind code	B2
Filing date	Dec 8, 2020
Priority date	Dec 8, 2020
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

A first classification is received from a neural network regarding a training dataset sent to the neural network. A modified training dataset with a perturbation of the training dataset is identified, where this modified training dataset causes the neural network to return a second classification. The perturbation is analyzed to identify a negative rule of the neural network.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving a first classification from a neural network regarding a training dataset sent to the neural network, wherein the neural network is configured to identify positive and negative evidence;
identifying a modified training dataset with a perturbation of the training dataset that causes the neural network to return a second classification that is different from the first classification; and
analyzing the perturbation to identify a negative rule of the neural network, wherein the negative rule relates to an absence of an element in an input of the training dataset increasing a likelihood that the neural network returns the first classification, wherein a presence of the element increases a likelihood that the neural network returns the second classification.

2. The computer-implemented method of claim 1 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying features that are present in the modified training dataset that are not in the training dataset.

3. The computer-implemented method of claim 2 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying that some of the features that are not in the training dataset are more relevant to the training dataset.

4. The computer-implemented method of claim 3 , wherein identifying that some of the features that are not in the training dataset are more relevant includes using term frequency inverse document frequency.

5. The computer-implemented method of claim 1 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying features of the training dataset that were most impacted by the perturbation.

6. The computer-implemented method of claim 1 , wherein the perturbation is determined using an adversarial attack method.

7. The computer-implemented method of claim 1 , wherein the perturbation is determined using a gradient-free optimization technique.

8. The computer-implemented method of claim 1 , further comprising interpreting output from the neural network using the negative rule.

9. The computer-implemented method of claim 1 , further comprising identifying the perturbation as a minimal perturbation to cause the neural network to return the second classification.

10. A system comprising:
a processor; and
a memory in communication with the processor, the memory containing instructions that, when executed by the processor, cause the processor to:
receive a first classification from a neural network regarding a training dataset sent to the neural network, wherein the neural network is configured to identify positive and negative evidence;
identify a modified training dataset with a perturbation of the training dataset that causes the neural network to return a second classification that is different from the first classification; and
analyze the perturbation to identify a negative rule of the neural network, wherein the negative rule relates to an absence of an element in an input of the training dataset increasing a likelihood that the neural network returns the first classification, wherein a presence of the element increases a likelihood that the neural network returns the second classification.

11. The system of claim 10 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying features that are present in the modified training dataset that are not in the training dataset.

12. The system of claim 11 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying that some of the features that are not in the training dataset are more relevant to the training dataset using term frequency inverse document frequency.

13. The system of claim 10 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying features of the training dataset that were most impacted by the perturbation to create the modified training dataset.

14. The system of claim 10 , wherein the perturbation is determined using an adversarial attack method.

15. The system of claim 10 , the memory containing additional instructions that, when executed by the processor, cause the processor to interpret output from the neural network using the negative rule.

16. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
receive a first classification from a neural network regarding a training dataset sent to the neural network, wherein the neural network is configured to identify positive and negative evidence;
identify a modified training dataset with a perturbation of the training dataset that causes the neural network to return a second classification that is different from the first classification; and
analyze the perturbation to identify a negative rule of the neural network, wherein the negative rule relates to an absence of an element in an input of the training dataset increasing a likelihood that the neural network returns the first classification, wherein a presence of the element increases a likelihood that the neural network returns the second classification.

17. The computer program product of claim 16 , wherein analyzing the perturbation to identify the negative rule of the neural network includes at least one of:
identifying features that are present in the modified training dataset that are not in the training dataset; and
identifying that some of the features that are not in the training dataset are more relevant to the training dataset using term frequency inverse document frequency.

18. The computer program product of claim 16 , wherein analyzing the perturbation to identify the negative rule of the neural network includes identifying features of the training dataset that were most impacted by the perturbation to create the modified training dataset.

19. The computer program product of claim 16 , wherein the perturbation is determined using an adversarial attack method.

20. The computer program product of claim 16 , the computer readable storage medium having additional program instructions embodied therewith that are executable by the computer to cause the computer to interpret output from the neural network using the negative rule.

Assignees

• IBM

Inventors

• Le Franck Vinh
• Srivatsa Mudhakar

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06F18/2148 Primary

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N5/025 Primary

  Extracting rules from data · CPC title