Device and method for training a normalizing flow

Patent metadata

Field	Value
Publication number	US-11961275-B2
Application number	US-202117402936-A
Country	US
Kind code	B2
Filing date	Aug 16, 2021
Priority date	Sep 4, 2020
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

A computer-implemented method for training a normalizing flow. The normalizing flow predicts a first density value based on a first input image. The first density value characterizes a likelihood of the first input image to occur. The first density value is predicted based on an intermediate output of a first convolutional layer of the normalizing flow. The intermediate output is determined based on a plurality of weights of the first convolutional layer. The method for training includes: determining a second input image; determining an output, wherein the output is determined by providing the second input image to the normalizing flow and providing an output of the normalizing flow as output; determining a second density value based on the output tensor and on the plurality of weights; determining a natural gradient of the plurality of weights with respect to the second density value; adapting the weights according to the natural gradient.

First claim

What is claimed is:

1. A computer-implemented method for training a normalizing flow, wherein the normalizing flow is configured to predict a first density value based on a first input image, wherein the first density value characterizes a likelihood of the first input image to occur, wherein the first density value is predicted based on an intermediate output of a first convolutional layer of the normalizing flow, and wherein the intermediate output is determined based on a plurality of weights of the first convolutional layer, the method for training comprising the following steps:
determining a second input image;
determining an output tensor, wherein the output is determined by providing the second input image to the normalizing flow and providing an output of the normalizing flow as the output tensor; determining a second density value based on the output tensor and on the plurality of weights;
determining a natural gradient of the plurality of weights with respect to the second density value; and adapting the plurality of weights according to the natural gradient;
wherein the natural gradient is determined according to the formula:
∇ w (l) =δ l *( w (l) * T h (l) ) T +w (l) ·H l ·W l ,

wherein ∇ w (l) is the natural gradient, δ l is an error signal for the first convolutional layer, w (l) is the plurality of weights, H l is a height of a layer input of the first convolutional layer, W l is a width of the layer input, * denotes a convolution operation and * T denotes a transposed convolution operation.

2. A computer-implemented method for training an image classifier, wherein the image classifier is configured to determine an output signal characterizing a classification of a first input image, the method comprising the following steps:
determining a training dataset, wherein the training dataset includes a plurality of second input images; training a normalizing flow using the training dataset, wherein the normalizing flow is configured to predict a first density value based on an input image, wherein the first density value characterizes a likelihood of the input image to occur, wherein the first density value is predicted based on an intermediate output of a first convolutional layer of the normalizing flow, and wherein the intermediate output is determined based on a plurality of weights of the first convolutional layer, the training of the normalizing flow including, for each second image of the second input images:
determining an output tensor, wherein the output is determined by providing the second input image to the normalizing flow and providing an output of the normalizing flow as the output tensor,
determining a second density value based on the output tensor and on the plurality of weights,
determining a natural gradient of the plurality of weights with respect to the second density value, and
adapting the plurality of weights according to the natural gradient;

providing the trained normalizing flow to the image classifier; providing the image classifier as a trained image classifier;
wherein the natural gradient is determined according to the formula:
∇ w (l) =δ l *( w (l) * T h (l) ) T +w (l) ·H l ·W l ,

wherein ∇ w (l) is the natural gradient, δ l is an error signal for the first convolutional layer, w (l) is the plurality of weights, H l is a height of a layer input of the first convolutional layer, W l is a width of the layer input, * denotes a convolution operation and * T denotes a transposed convolution operation.

3. The method according to claim 2 , wherein the training dataset further includes for each of the second input images a corresponding desired output signal, wherein the desired output signal characterizes a classification of the corresponding second input image, and the method further comprises the following steps:
splitting the training dataset into a plurality of subsets, wherein each subset includes the second input images that correspond with the desired output signals that characterize the same class;
training a respective normalizing flow for each of the subsets, wherein each respective normalizing flow corresponds to the class characterized by the corresponding output signals of the second input images the normalizing flow is trained with; and
providing the trained normalizing flows to the image classifier.

4. A computer-implemented method for classifying a first input image using an image classifier, wherein the image classifier provides an output signal characterizing a classification of the first input image, the method comprising the following steps of:
training the image classifier, the training of the imaging classifier including:
determining a training dataset, wherein the training dataset includes a plurality of second input images; training a normalizing flow using the training dataset, wherein the normalizing flow is configured to predict a first density value based on an input image, wherein the first density value characterizes a likelihood of the input image to occur, wherein the first density value is predicted based on an intermediate output of a first convolutional layer of the normalizing flow, and wherein the intermediate output is determined based on a plurality of weights of the first convolutional layer, the training of the normalizing flow including, for each second image of the second input images:
determining an output tensor, wherein the output is determined by providing the second input image to the normalizing flow and providing an output of the normalizing flow as the output tensor,
determining a second density value based on the output tensor and on the plurality of weights,
determining a natural gradient of the plurality of weights with respect to the second density value, and
adapting the plurality of weights according to the natural gradient;
providing the trained normalizing flow to the image classifier;
providing the image classifier as a trained image classifier;
predicting a first density value for the first input image using the trained normalizing flow from the image classifier;
providing the output signal such that the output signal characterizes a first class when the first density value is below than a predefined threshold;

providing the output signal such that the output signal characterizes a second class when the first density value is equal to the predefined threshold or above the predefined threshold;
wherein the natural gradient is determined according to the formula:
∇ w (l) =δ l *( w (l) * T h (l) ) T +w (l) ·H l ·W l ,

wherein ∇ w (l) is the natural gradient, δ l is an error signal for the first convolutional layer, w (l) is the plurality of weights, H l is a height of a layer input of the first convolutional layer, W l is a width of the layer input, * denotes a convolution operation and * T denotes a transposed convolution operation.

5. The method as recited in claim 4 , wherein a device is operated based on the output signal.

6. A computer-implemented method for classifying a first input image using an image classifier, wherein the image classifier provides an output signal characterizing a classification of the first input image, the method comprising the following steps:
training the image classifier wherein the image classifier is configured to determine an output signal characterizing a classification of a first input image, the image classifier being trained by performing the following steps:
determining a training dataset, wherein the training dataset includes a plurality of second input images, and wherein the training dataset further includes for each of the second input images a corresponding desired output signal, wherein the desi

Assignees

• Bosch Gmbh Robert

Inventors

• Peters Jorn
• Keller Thomas Andy
• Khoreva Anna
• Hoogeboom Emiel
• Welling Max
• Jaini Priyank

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/32 Primary

  Normalisation of the pattern dimensions · CPC title