Using a neural network

Patent metadata

Field	Value
Publication number	US-11468323-B2
Application number	US-201816756182-A
Country	US
Kind code	B2
Filing date	Oct 16, 2018
Priority date	Oct 19, 2017
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

A method, system and computer-program product for identifying neural network inputs for a neural network that may have been incorrectly processed by the neural network. A set of activation values (of a subset of neurons of a single layer) associated with a neural network input is obtained. A neural network output associated with the neural network input is also obtained. A determination is made as to whether a first and second neural network input share similar sets of activation values, but dissimilar neural network outputs or vice versa. In this way a prediction can be made as to whether one of the first and second neural network inputs has been incorrectly processed by the neural network.

First claim

The invention claimed is:

1. A computer implemented method for using a neural network formed of one or more layers of neurons, the method comprising:
using the neural network to process a first neural network input to produce a first neural network output;
using the neural network to process a second neural network input to produce a second neural network output;
determining a first similarity indicator, the first similarity indicator indicating a level of similarity between the first neural network output and the second neural network output;
selecting a set of one or more neurons of the neural network, the set of neurons being a subset of the neurons of a single layer of the neural network;
determining an activation value of each selected neuron when the neural network is used to process the first neural network input, to thereby produce a first set of activation values;
determining an activation value of each selected neuron when the neural network is used to process the second neural network input, to thereby produce a second set of activation values;
determining a second similarity indicator indicating a level of similarity between the first set of activation values and a corresponding second set of activation values;
determining a potential inaccurate processing by the neural network of one of the first and second neural network inputs, the potential inaccurate processing comprising either:
the first similarity indicator indicating a level of similarity above a first predetermined threshold and the second similarity indicator indicating a level of similarity below a second predetermined threshold; or
the first similarity indicator indicating a level of similarity below a third predetermined threshold and the second similarity indicator indicating a level of similarity above a fourth predetermined threshold, and

flagging the potential inaccurate processing to a user.

2. The computer-implemented method of claim 1 , wherein:
the step of using the neural network to process a first neural network input comprises classifying the first neural network input into a classification, the classification being the first neural network output;
the step of using the neural network to process a second neural network input comprises classifying the second neural network input into a classification, the classification being the second neural network output;
the step of determining a first similarity indicator comprises generating a first similarity indicator identifying whether the classifications of the first and second neural network inputs are the same; and
the step of determining a potential inaccurate processing by the neural network of whether one of the first and second neural network inputs comprises determining whether either:
the first similarity indicator indicates that the classifications of the first and second neural network inputs are the same and the second similarity indicator indicates a level of similarity below a second predetermined threshold; or
the first similarity indicator indicates that the classifications of the first and second neural network inputs are different and the second similarity indicator indicates a level of similarity above a fourth predetermined threshold.

3. The computer-implemented method of claim 1 , wherein the set of neurons comprises at least two neurons of the same layer.

4. The computer-implemented method of claim 3 , wherein the step of determining a second similarity indicator comprises calculating a multidimensional distance between the first set of values and the second set of values.

5. The computer-implemented method of claim 1 , wherein the second and fourth predetermined thresholds are predetermined threshold distances.

6. The computer-implemented method of claim 1 , wherein the step of selecting a set of one or more neurons comprises selecting one or more neurons in a same layer having activation values lying outside a predetermined range of activation values when the neural network is used to process the first neural network input.

7. The computer-implemented method of claim 1 , wherein the step of selecting a set of one or more neurons comprises selecting one or more neurons in a same layer having activation values lying outside a predetermined range of activation values when the neural network is used to process the second neural network input.

8. The computer-implemented method of claim 1 , wherein:
the step of using the neural network to process a second neural network input comprises using the neural network to process a plurality of second neural network inputs to thereby generate a plurality of second neural network outputs;
the step of determining a first similarity indicator comprises determining, for each of the plurality of second neural network inputs, a first similarity indicator indicating a level of similarity between the first neural network output and the second neural network output to thereby produce a plurality of first similarity indicators;
the step of producing a second set of activation values comprises determining, for each of the plurality of second neural network inputs, an activation value of each selected neuron when the neural network is used to process the second neural network input, to thereby produce a second set of activation values for each second neural network input and thereby a plurality of second sets of activation values;
the step of determining a second similarity indicator comprises determining, for each of the plurality of second neural network inputs, a second similarity indicator indicating a level of similarity between the first set of activation values and a corresponding second set of activation values to thereby produce a plurality of second similarity indicators; and
the step of determining a potential inaccurate processing by the neural network of one of the first and second neural network inputs comprises:
determining, for each second neural network input, whether either the first similarity indicator indicates a level of similarity above a first predetermined threshold and the second similarity indicator indicates a level of similarity below a second predetermined threshold; or the first similarity indicator indicates a level of similarity below a third predetermined threshold and the second similarity indicator indicates a level of similarity above a fourth predetermined threshold, to thereby determine, for each second neural network input, whether one of the first and second neural network inputs has been potentially inaccurately processed by the neural network; and

identifying all second neural network inputs for which it is determined that one of the first and second neural network inputs has been potentially inaccurately processed by the neural network.

9. The method of claim 8 , wherein the step of determining a second similarity indicator for each second neural network input comprises:
performing multidimensional clustering on the first set of activation values and each of the plurality of second set of activation values; and
generating, for each second neural network input, a second similarity indicator indicating whether the first neural network input and the second neural network inputs are associated with sets of activation values in a same cluster.

10. A computer implemented method as in a claim 1 , further comprising obtaining a user input indicating a correct neural network output for one or more the first and second neural network input, and re-training the neural network based on the correct neural network output.

11. A computer program product comprising a non-transitory computer readable medium, the computer readable medium having computer readable code e

Assignees

• Koninklijke Philips Nv

Inventors

• Menkovski Vlado
• Rahman Asif
• Raynaud Caroline Denise Francoise
• Conroy Bryan
• Mavroeidis Dimitrios
• Bresch Erik
• Van Den Heuvel Teun

Classifications

• G06N3/02 Primary

  Neural networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06F18/285

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title