Configuring a neural network for equivariant or invariant behavior

Patent metadata

Field	Value
Publication number	US-12164302-B2
Application number	US-202217869438-A
Country	US
Kind code	B2
Filing date	Jul 20, 2022
Priority date	Aug 10, 2021
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

A method for configuring a neural network which is designed to map measured data to one or more output variables. The method includes: transformation(s) of the measured data is/are specified which when applied to the measured data, is/are meant to induce the output variables supplied by the neural network to exhibit an invariant or equivariant behavior; at least one equation is set up which links a condition that the desired invariance or equivariance be given with the architecture of the neural network; by solving the at least one equation a feature is obtained that characterizes the desired architecture and/or a distribution of weights of the neural network in at least one location of this architecture; a neural network is configured in such a way that its architecture and/or its distribution of weights in at least one location of this architecture has/have all of the features ascertained in this way.

First claim

What is claimed is:

1. A method for configuring a neural network which is configured to map measured data to one or more output variables, the method comprising the following steps:
specifying one or more transformations of the measured data which, when applied to the measured data, is meant to induce output variables supplied by the neural network to exhibit a desired invariant or equivariant behavior;
setting up at least one equation which links a condition that the desired invariance or equivariance be given with an architecture of the neural network;
obtaining, by solving the at least one equation, at least one feature that characterizes the architecture and/or a distribution of weights of the neural network in at least one location of the architecture; and
configuring the neural network in such a way that its architecture and/or the distribution of weights in at least one location of the architecture, has all of the ascertained at least one feature;
wherein:
observations of multiple agents of a centralized or decentralized Markov decision process are the measured data,
a reward to be expected when a predefined action is performed in a certain state of the Markov decision process and/or a policy for at least one agent mapping a predefined state to an action to be performed is selected as an output variable of the output variables; and

wherein the method further comprises:
ascertaining, from the reward to be expected and/or from the policy, a control signal for at least one robot and/or for at least one vehicle and/or for at least one unmanned flying device, and
controlling, using the control signal, the robot and/or the vehicle and/or the unmanned flying device.

2. The method as recited in claim 1 , wherein the at least one equation includes:
a function ϕ u , which describes a further development of features of layers of the neural network during a transition from one layer to the next, and/or
a function ϕ m , which describes an information flow within the neural network as a function of the architecture of the neural network.

3. The method as recited in claim 2 , wherein the neural network is a graph in which nodes occupied by features h i l are connected by edges e ij .

4. The method as recited in claim 3 , wherein the function du links features h i l+1 of the i th node in a layer l+1 with features h i l of the i th node in a layer I and with the information flow m i l received in total by the node.

5. The method as recited in claim 4 , wherein the function Om links the information flow m j→i l from node j to node i in a layer l with an edge e ij between nodes I and j and also with features h j l of the j th node in the layer I.

6. The method as recited in claim 1 , wherein, in the specifying step, at least one group of transformations is specified for which the desired invariance or equivariance of the output variables is to apply.

7. The method as recited in claim 1 , wherein the at least one equation is expressed in hyperparameters which characterize the architecture of the neural network, and the solving of the at least one equation leads to values of the hyperparameters as features.

8. The method as recited in claim 1 , wherein the observations include positions of the agents.

9. A method for configuring a neural network which is configured to map measured data to one or more output variables, the method comprising the following steps:
specifying one or more transformations of the measured data which, when applied to the measured data, is meant to induce output variables supplied by the neural network to exhibit a desired invariant or equivariant behavior;
setting up at least one equation which links a condition that the desired invariance or equivariance be given with an architecture of the neural network;
obtaining, by solving the at least one equation, at least one feature that characterizes the architecture and/or a distribution of weights of the neural network in at least one location of the architecture; and
configuring the neural network in such a way that its architecture and/or the distribution of weights in at least one location of the architecture, has all of the ascertained at least one feature;
wherein the neural network to be configured is a classifier network, which maps the measured data to classification scores with regard to one or more classes of a predefined classification; and
wherein the method further comprises:
ascertaining, from the classification scores, a control signal for at least one robot and/or for at least one vehicle and/or for at least one unmanned flying device, and
controlling, using the control signal, the robot and/or the vehicle and/or the unmanned flying device.

10. A non-transitory machine-readable data carrier on which is stored a computer program for configuring a neural network which is configured to map measured data to one or more output variables, the computer program, when executed by one or more computers, causing the one or more computers to perform the following steps:
specifying one or more transformations of the measured data which, when applied to the measured data, is meant to induce output variables supplied by the neural network to exhibit a desired invariant or equivariant behavior;
setting up at least one equation which links a condition that the desired invariance or equivariance be given with an architecture of the neural network;
obtaining, by solving the at least one equation, at least one feature that characterizes the architecture and/or a distribution of weights of the neural network in at least one location of the architecture; and
configuring the neural network in such a way that its architecture and/or the distribution of weights in at least one location of the architecture, has all of the ascertained at least one feature;
wherein the neural network to be configured is a classifier network, which maps the measured data to classification scores with regard to one or more classes of a predefined classification; and
wherein the computer program, when executed by the one or more computers, further causing the one or more computer to perform the following steps:
ascertaining, from the classification scores, a control signal for at least one robot and/or for at least one vehicle and/or for at least one unmanned flying device, and
controlling, using the control signal, the robot and/or the vehicle and/or the unmanned flying device.

11. One or more computers configured to configure a neural network which is configured to map measured data to one or more output variables, the one or more computers being configured to:
specify one or more transformations of the measured data which, when applied to the measured data, is meant to induce output variables supplied by the neural network to exhibit a desired invariant or equivariant behavior;
set up at least one equation which links a condition that the desired invariance or equivariance be given with an architecture of the neural network;
obtain, by solving the at least one equation, at least one feature that characterizes the architecture and/or a distribution of weights of the neural network in at least one location of the architecture; and
configure the neural network in such a way that its architecture and/or the distribution of weights in at least one location of the architecture, has all of the ascertained at least one feature;
wherein the neural network to be configured is a classifier network, which maps the measured data to classification scores with regard to one or more classes of a predefined classification; and
wherein the one or more computers are further configured to:
ascertain, from the classification sc

Assignees

• Bosch Gmbh Robert

Inventors

• Van Der Pol Elise
• Oliehoek Frans A
• Van Hoof Herke
• Welling Max
• Herman Michael

Classifications

• G06N3/08

  Learning methods · CPC title

• G05D1/46

  Control of position or course in three dimensions [3D] · CPC title

• B25J9/161

  Hardware, e.g. neural networks, fuzzy logic, interfaces, processor · CPC title

• G05D1/101

  specially adapted for aircraft · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title