Compressing images using neural networks

Patent metadata

Field	Value
Publication number	US-11336908-B2
Application number	US-201916586837-A
Country	US
Kind code	B2
Filing date	Sep 27, 2019
Priority date	Feb 5, 2016
Publication date	May 17, 2022
Grant date	May 17, 2022

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressing images using neural networks. One of the methods includes receiving an image; processing the image using an encoder neural network, wherein the encoder neural network is configured to receive the image and to process the image to generate an output defining values of a first number of latent variables that each represent a feature of the image; generating a compressed representation of the image using the output defining the values of the first number of latent variables; and providing the compressed representation of the image for use in generating a reconstruction of the image.

First claim

What is claimed is:

1. A method comprising:
receiving a lossy compressed representation of an image, wherein the lossy compressed representation of the image defines values of a first number of latent variables that each represent a feature of the image; and
generating a reconstruction of the image from the lossy compressed representation of the image, comprising:
selecting a value of one or more additional latent variables that are not in the first number of latent variables randomly from a prior distribution; and
generating the reconstruction of the image by conditioning a generative neural network on the values of the first number of latent variables and the randomly selected values of the additional latent variables that are not in the first number of latent variables, wherein:

the generative neural network has been trained jointly with an encoder neural network as a variational auto-encoder network that includes the encoder neural network and the generative neural network, and the encoder neural network is configured to process the image to generate an encoder output that define parameters of statistical distributions of the first number of latent variables and the additional latent variables.

2. The method of claim 1 , wherein generating the reconstruction comprises:
decompressing the values of the first number of latent variables from the lossy compressed representation.

3. The method of claim 1 , wherein receiving the lossy compressed representation comprises:
retrieving the lossy compressed representation from memory.

4. The method of claim 1 , wherein receiving the lossy compressed representation comprises:
receiving the lossy compressed representation over a network from an encoder system.

5. The method of claim 1 , further comprising generating the lossy compressed representation of the image, comprising:
processing the image using the encoder neural network, wherein the encoder neural network is configured to receive the image and to process the image to generate an output defining values of a second number of latent variables that is greater than the first number; and
generating the lossy compressed representation of the image using the first number of the latent variables.

6. The method of claim 1 , wherein the features are arranged in a hierarchy from least abstract representation of the image to most abstract representation of the image, and wherein the first number of latent variables are the latent variables that represent features at a predetermined number of highest levels in the hierarchy.

7. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
receiving a lossy compressed representation of an image, wherein the lossy compressed representation of the image defines values of a first number of latent variables that each represent a feature of the image; and
generating a reconstruction of the image from the lossy compressed representation of the image, comprising:
selecting a value of one or more additional latent variables that are not in the first number of latent variables randomly from a prior distribution; and
generating the reconstruction of the image by conditioning a generative neural network on the values of the first number of latent variables and the randomly selected values of the additional latent variables that are not in the first number of latent variables, wherein:

the generative neural network has been trained jointly with an encoder neural network as a variational auto-encoder network that includes the encoder neural network and the generative neural network, and the encoder neural network is configured to process the image to generate an encoder output that define parameters of statistical distributions of the first number of latent variables and the additional latent variables.

8. The system of claim 7 , wherein generating the reconstruction comprises:
decompressing the values of the first number of latent variables from the lossy compressed representation.

9. The system of claim 7 , wherein receiving the lossy compressed representation comprises:
retrieving the lossy compressed representation from memory.

10. The system of claim 7 , wherein receiving the lossy compressed representation comprises:
receiving the lossy compressed representation over a network from an encoder system.

11. The system of claim 7 , the operations further comprising generating the lossy compressed representation of the image, comprising:
processing the image using the encoder neural network, wherein the encoder neural network is configured to receive the image and to process the image to generate an output defining values of a second number of latent variables that is greater than the first number; and
generating the lossy compressed representation of the image using the first number of the latent variables.

12. The system of claim 7 , wherein the features are arranged in a hierarchy from least abstract representation of the image to most abstract representation of the image, and wherein the first number of latent variables are the latent variables that represent features at a predetermined number of highest levels in the hierarchy.

13. One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving a lossy compressed representation of an image, wherein the lossy compressed representation of the image defines values of a first number of latent variables that each represent a feature of the image; and
generating a reconstruction of the image from the lossy compressed representation of the image, comprising:
selecting a value of one or more additional latent variables that are not in the first number of latent variables randomly from a prior distribution; and
generating the reconstruction of the image by conditioning a generative neural network on the values of the first number of latent variables and the randomly selected values of the additional latent variables that are not in the first number of latent variables, wherein:

the generative neural network has been trained jointly with an encoder neural network as a variational auto-encoder network that includes the encoder neural network and the generative neural network, and the encoder neural network is configured to process the image to generate an encoder output that define parameters of statistical distributions of the first number of latent variables and the additional latent variables.

14. The computer-readable storage media of claim 13 , wherein generating the reconstruction comprises:
decompressing the values of the first number of latent variables from the lossy compressed representation.

15. The computer-readable storage media of claim 13 , wherein receiving the lossy compressed representation comprises:
retrieving the lossy compressed representation from memory.

16. The computer-readable storage media of claim 13 , wherein receiving the lossy compressed representation comprises:
receiving the lossy compressed representation over a network from an encoder system.

Assignees

• Deepmind Tech Ltd

Inventors

• Wierstra Daniel Pieter
• Gregor Karol
• Besse Frederic Olivier

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title