Verification of perception systems

Patent metadata

Field	Value
Publication number	US-12547879-B2
Application number	US-201917295752-A
Country	US
Kind code	B2
Filing date	Nov 26, 2019
Priority date	Nov 26, 2018
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

There is provided a computer-implemented method for verifying the robustness of a neural network classifier with respect to one or more parameterised transformations applied to an input, the classifier comprising one or more convolutional layers, the method comprising: encoding each layer of the classifier as one or more algebraic classifier constraints; encoding each transformation as one or more algebraic transformation constraints; encoding a change in an output classifier label from the classifier as an algebraic output constraint; determining whether a solution exists which satisfies the classifier constraints, transformation constraints and output constraints, and determining the classifier as robust to the local transformations if no such solution exists. A perception system and a computer readable medium are also provided.

First claim

The invention claimed is:

1 . A computer-implemented method for operating an actuator using a neural network image classifier having robustness verified with respect to one or more parameterized transformations applied to an input image, the classifier comprising one or more convolutional layers, the method comprising:
encoding each layer of the image classifier as one or more algebraic classifier constraints, wherein the image classifier is configured to receive the input image and output a classifier label; encoding each parameterized transformation as one or more algebraic transformation constraints, the parametrized transformations including at least one of translation, rotation scaling, shear, brightness and/or contrast; encoding a change in an output classifier label from the image classifier as an algebraic output constraint; determining whether a solution exists which satisfies the classifier constraints, transformation constraints and output constraints, and determining the classifier as robust to the parameterized transformations if no such solution exists such that the image classifier always returns the same classifier label for an image of a given scene regardless of the application of the one or more parameterized transformations; the method further comprising, where a solution exists which satisfies the classifier constraints, transformation constraints and output constraints: identifying parameters of the one or more transformations associated with the solution; generating additional training data by applying the one or more transformations to existing training data using the identified parameters; and training the classifier using the additional training data and the method further comprises: operating the classifier to classify image data obtained from a camera, generating a control signal in dependence on an output of the classifier, and operating an actuator in accordance with the control signal.

2 . The computer-implemented method according to claim 1 , wherein one or more of the classifier, transformation and output constraints are linear constraints.

3 . The computer-implemented method according to claim 1 , wherein one or more of the classifier, transformation and output constraints are non-linear constraints.

4 . The computer-implemented method according to claim 1 , wherein the classifier further comprises one or more fully connected layers.

5 . The computer-implemented method according to claim 1 , wherein encoding each layer of the classifier as one or more algebraic classifier constraints comprises deriving a mixed-integer linear programming expression for each layer.

6 . The computer-implemented method according to claim 1 , wherein encoding each transformation as one or more algebraic transformation constraints comprises deriving a mixed-integer linear programming expression for each layer.

7 . The computer-implemented method according to claim 1 , wherein encoding each transformation as one or more algebraic transformation constraints comprises deriving a mixed-integer non-linear programming expression for each layer.

8 . The computer-implemented method according to claim 1 , wherein one or more of the classifier layers comprises a rectified linear unit activating function.

9 . A perception system comprising a neural network image classifier implemented on one or more processors configured to carry out a method for verifying the robustness of the neural network image classifier with respect to one or more parameterized transformations applied to an input image, the classifier including one or more convolutional layers, the method including:
encoding each layer of the image classifier as one or more algebraic classifier constraints, wherein the image classifier is configured to receive the input image and output a classifier label; encoding each parameterized transformation as one or more algebraic transformation constraints, the parametrized transformations including at least one of translation, rotation scaling, shear, brightness and/or contrast; encoding a change in an output classifier label from the image classifier as an algebraic output constraint; determining whether a solution exists which satisfies the classifier constraints, transformation constraints and output constraints, and determining the classifier as robust to the parameterized transformations if no such solution exists such that the image classifier always returns the same classifier label for an image of a given scene regardless of the application of the one or more parameterized transformations; the method further comprising, where a solution exists which satisfies the classifier constraints, transformation constraints and output constraints: identifying parameters of the one or more transformations associated with the solution; generating additional training data by applying the one or more transformations to existing training data using the identified parameters; and training the classifier using the additional training data, wherein the perception system further comprises a controller and an actuator, the classifier is configured to classify image data received from a camera, the controller is configured to generate a control signal in dependence on an output of the classifier, and the actuator is configured to operate in accordance with the control signal received from the controller.

Assignees

• Imperial College Sci Tech & Medicine
• Imperial College Innovations Ltd

Inventors

• Lomuscio Alessio
• Kouvaros Panagiotis

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/094

  Adversarial learning · CPC title