Image classifier comprising a non-injective transformation

Patent metadata

Field	Value
Publication number	US-12450891-B2
Application number	US-202117345702-A
Country	US
Kind code	B2
Filing date	Jun 11, 2021
Priority date	Jul 3, 2020
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

A computer-implemented method of training an image classifier which uses any combination of labelled and/or unlabelled training images. The image classifier comprises a set of transformations between respective transformation inputs and transformation outputs. An inverse model is defined in which for a deterministic, non-injective transformation of the image classifier, its inverse is approximated by a stochastic inverse transformation. During training, for a given training image, a likelihood contribution for this transformation is determined based on a probability of its transformation inputs being generated by the stochastic inverse transformation given its transformation outputs. This likelihood contribution is used to determine a log-likelihood for the training image to be maximized (and its label, if the training image is labelled), based on which the model parameters are optimized.

First claim

What is claimed is:

1. A computer-implemented method of training an image classifier, the image classifier being configured to classify an input image into a class from a set of classes, the method comprising the following steps:
accessing a training dataset, the training dataset including at least one labelled training image labelled with a training class from the set of classes and at least one unlabelled training image;
defining an inverse model for the image classifier, the inverse model configured to map output classes of the image classifier to input images, wherein the image classifier includes a set of transformations, the set of transformations including at least one deterministic and non-injective transformation, an inverse of the deterministic and non-injective transformation being approximated in the inverse model by a stochastic inverse transformation, the inverse model including trainable parameters, and sharing a plurality of the trainable parameters with the image classifier;
training the image classifier using a log-likelihood optimization, the training including:
selecting a training image from the training dataset,
applying the image classifier to the training image, including applying the deterministic and non-injective transformation to transformation inputs of the deterministic and non-injective transformation to obtain transformation outputs of the deterministic and non-injective transformation,
determining a likelihood contribution for the deterministic and non-injective transformation of the image classifier based on a probability that the stochastic inverse transformation of the inverse model generates the transformation inputs given the transformation outputs,
when the training image is the labelled training image, using the likelihood contribution to determine a log-likelihood for the labelled training image and its label according to a joint probability distribution of input images and classes determined by the image classifier,
when the training image is the unlabelled training image, using the determined likelihood contribution to determine a log-likelihood for the unlabelled training image according to a probability distribution of input images being generated by the inverse model; and
optimizing parameters of the image classifier to maximize the log-likelihood for the labelled training image occurring according to the joint probability distribution and optimizing parameters of the inverse model to maximize the log-likelihood for the unlabelled training image for the unlabelled training image occurring according to the probability distribution of input images being generated by the inverse model.

2. The method of claim 1 , wherein the determining of the log-likelihood for the training image includes determining a sum of likelihood contributions for respective transformations of the set of transformations.

3. The method of claim 2 , wherein the image classifier includes a densely connected component given by a linear bijective transformation and a slicing transformation, wherein the slicing transformation is configured to select a subset of outputs of the linear bijective transformation, wherein an inverse of the slicing transformation is approximated in the inverse model by a stochastic inverse transformation configured to sample non-selected outputs for the linear bijective transformation based on the selected outputs of the linear bijective transformation.

4. The method of claim 2 , wherein the image classifier includes a coupling transformation configured to determine first and second transformation outputs given first and second transformation inputs by combining the first transformation input with a first function of the second transformation input to obtain the first transformation output, and combining the second transformation input with a second function of the first transformation output, wherein the first and second functions are convolutions.

5. The method of claim 1 , wherein the image classifier includes a max pooling transformation computing a transformation output as a maximum of multiple transformation inputs, wherein an inverse of the max pooling transformation is approximated in the inverse model by an inverse transformation configured to sample an index of a maximal transformation input and values of non-maximal transformation inputs.

6. The method of claim 1 , wherein the image classifier includes a ReLU transformation configured to compute a transformation output by mapping a transformation input from an interval to a constant, wherein an inverse of the ReLU transformation is approximated in the inverse model by an inverse transformation configured to, given a transformation output equal to the given constant, sample a transformation input from the given interval.

7. The method of claim 1 , wherein the image classifier is configured to classify the input image into the class by determining a vector of class probabilities for respective classes, and, in an output layer, determining the class from the vector, wherein an inverse of the output layer is approximated in the inverse model based on a conditional probability distribution for the vector of class probabilities given the determined class.

8. The method of claim 1 , wherein the image classifier further includes a stochastic transformation with a deterministic inverse transformation, and the method further comprises computing a likelihood contribution of the stochastic transformation based on a probability that the deterministic and non-injective transformation generates the transformation outputs of the stochastic transformation given the transformation inputs of the stochastic transformation.

9. The method of claim 1 , further comprising:
obtaining an input image data using a sensor of an at least semi-autonomous vehicle;
classifying the input image data using the trained image classifier, including applying the trained image classifier to classify the input image into a class from the set of classes; and
controlling steering and/or braking of the vehicle based on the classification of the input image data.

10. The method of claim 1 , further comprising:
obtaining a target class;
applying the trained inverse model to the target class to generate, using the trained inverse model, multiple synthetic images representative of the target class; and
further training the trained image classifier using the generated synthetic images.

11. The method of claim 1 , further comprising:
obtaining a target class;
applying the trained inverse model to the target class to generate, using the trained inverse model, multiple synthetic images representative of the target class; and
training a machine learning model using the generated synthetic images.

12. A computer-implemented method of using a trained image classifier, the trained image classifier being configured to classify an input image into a class from a set of classes, the method comprising the following steps:
accessing model data representing the trained image classifier, wherein the image classifier is trained by:
accessing a training dataset, the training dataset including at least one labelled training image labelled with a training class from the set of classes and at least one unlabelled training image,
defining an inverse model for the image classifier, the inverse model configured to map output classes of the image classifier to input images, wherein the image classifier includes a set of transformations, the set of transformations including at least one deterministic and non-injective transformation, an inverse of the deterministic and non-injective transformation being approximated in the inverse model by

Assignees

• Bosch Gmbh Robert

Inventors

• Nielsen Didrik
• Hoogeboom Emiel
• Sakmann Kaspar
• Welling Max
• Jaini Priyank

Classifications

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/048

  Activation functions · CPC title

• G06F18/2431

  Multiple classes · CPC title