Image generation model based on log-likelihood

Patent metadata

Field	Value
Publication number	US-11995151-B2
Application number	US-202117445891-A
Country	US
Kind code	B2
Filing date	Aug 25, 2021
Priority date	Sep 29, 2020
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

A computer-implemented method of training an image generation model. The image generation model comprises an argmax transformation configured to compute a discrete index feature indicating an index of a feature of the continuous feature vector with an extreme value. The image generation model is trained using a log-likelihood optimization. This involves obtaining a value of the index feature for the training image, sampling values of the continuous feature vector given the value of the index feature according to a stochastic inverse transformation of the argmax transformation, and determining a likelihood contribution of the argmax transformation for the log-likelihood based on a probability that the stochastic inverse transformation generates the values of the continuous feature vector given the value of the index feature.

First claim

What is claimed is:

1. A computer-implemented method of training an image generation model, the image generation model being configured to generate an image from a latent feature representation, the method comprising the following steps:
accessing model data representing parameters of the image generation model, and training data representing a training dataset of multiple training images;
training the image generation model using a log-likelihood optimization, wherein:
the training includes selecting a training image of the training images, and determining a log-likelihood of the training image being generated according to the image generation model,
the image generation model includes a transformation configured to determine a discrete feature from a continuous feature vector, the transformation being an argmax transformation configured to compute a discrete index feature indicating an index of a feature of the continuous feature vector with an extreme value, and
determining the log-likelihood includes obtaining a value of the index feature for the training image, sampling values of the continuous feature vector given the value of the index feature according to a stochastic inverse transformation of the argmax transformation, and determining a likelihood contribution of the argmax transformation for the log-likelihood based on a probability that the stochastic inverse transformation generates values of the continuous feature vector given the value of the index feature; and

outputting the trained image generation model.

2. The method of claim 1 , wherein the stochastic inverse transformation is parameterized by parameters included in the parameters of the image generation model.

3. The method of claim 2 , wherein sampling the values of the continuous feature vector given the value of the index feature includes sampling an initial feature vector, and applying an injective transformation to the initial feature vector based on the value of the index feature to obtain the values of the continuous feature vectors, the injective transformation being defined such that the index feature indicates an index of a feature of the continuous feature vector with an extreme value.

4. The method of claim 3 , wherein applying the injective transformation includes: (i) applying a smooth thresholding function to the feature of the continuous feature vector indicated by the index feature to make the feature indicated by the index feature larger or smaller than one or more values of the continuous feature vector, and/or (ii) applying a smooth thresholding function to one or more other values of the continuous feature vector to make the one or more other values smaller or larger than the feature indicated by the index feature.

5. The method of claim 2 , wherein sampling the values of the continuous feature vector given the value of the index feature includes:
sampling the value of the continuous feature vector indicated by the index feature according to a Gumbel distribution; and
sampling a value of the continuous feature vector not indicated by the index feature according to a truncated Gumbel distribution based on the sampled value indicated by the index feature.

6. The method of claim 1 , wherein the index feature corresponds to a particular pixel of the training image.

7. The method of claim 1 , wherein the image generation model is configured to determine a discrete feature by computing multiple respective discrete index features using the argmax transformation and combining the multiple discrete index features.

8. A computer-implemented method of using a trained image generation model, comprising the following steps:
accessing model data representing parameters of an image generation model, the image generation model including a transformation configured to determine a discrete feature from a continuous feature vector, the transformation being an argmax transformation configured to compute a discrete index feature indicating an index of a feature of the continuous feature vector with an extreme value, the image generation model having been trained on a training dataset, an inverse of the argmax transformation being approximated by a stochastic inverse transformation; and
applying the image generation model to a latent feature representation to obtain a generated image, and/or using the image generation model to determine a conformance value indicating a conformance of an input image to the training dataset, the conformance value being based on a log-likelihood of the input image being generated according to the image generation model and being computed using the stochastic inverse transformation.

9. The method of claim 8 , further comprising accessing model data representing an image classifier trained on the same training dataset as the image generation model, and using the conformance value as a reliability measure of the image classifier for the input image.

10. The method of claim 9 , wherein the input image is an image of an environment of a computer-controlled system, the method further comprising obtaining the input image from a camera of the system and, at least if the conformance value indicates a sufficient reliability of the image classifier for the input image, using an output of the image classifier on the input image to control said system.

11. The method of claim 8 , further comprising applying the image generation model repeatedly to obtain multiple generated images, and using the multiple generated images as training data to train a further machine learning model.

12. The method of claim 11 , further comprising obtaining an input instance to the further machine learning model and applying the further machine learning model to the input instance to determine an output of the further machine learning model.

13. A system for training an image generation model, the image generation model being configured to generate an image from a latent feature representation, the system comprising:
a data interface configured to access model data representing parameters of the image generation model, and training data representing a training dataset of multiple training images; and
a processor subsystem configured to train the image generation model using a log-likelihood optimization and output the trained image generation model, wherein:
the training includes selecting a training image of the training images, and determining a log-likelihood of the training image being generated according to the image generation model,
the image generation model includes a transformation configured to determine a discrete feature from a continuous feature vector, the transformation being an argmax transformation configured to compute a discrete index feature indicating an index of a feature of the continuous feature vector with an extreme value, and
determining the log-likelihood includes obtaining a value of the index feature for the training image, sampling values of the continuous feature vector given the value of the index feature according to a stochastic inverse transformation of the argmax transformation, and determining a likelihood contribution for the argmax transformation based on a probability that the stochastic inverse transformation generates values of the continuous feature vector given the value of the index feature.

14. A system for using a trained image generation model, comprising:
a data interface configured to access model data representing parameters of an image generation model, the image generation model including a transformation configured to determine a discrete feature from a continuous feature vector, the transformation being an argmax

Assignees

• Bosch Gmbh Robert

Inventors

• Hoogeboom Emiel
• Nielsen Didrik
• Welling Max
• Forre Patrick
• Jaini Priyank
• Beluch William Harris

Classifications

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06F18/214 Primary

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/213

  Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods · CPC title

• G06N20/00

  Machine learning · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title