Content adaptive optimization for neural data compression

Patent metadata

Field	Value
Publication number	US-11057634-B2
Application number	US-201916413414-A
Country	US
Kind code	B2
Filing date	May 15, 2019
Priority date	May 15, 2019
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

A data processing system includes a computing platform having a hardware processor and a memory storing a data compression software code. The hardware processor executes the data compression software code to receive a series of compression input data and encode a first compression input data of the series to a latent space representation of the first compression input data. The data compression software code further decodes the latent space representation to produce an input space representation of the first compression input data corresponding to the latent space representation, and generates f refined latent values for re-encoding the first compression input data based on a comparison of the first compression input data with its input space representation. The data compression software code then re-encodes the first compression input data using the refined latent values to produce a first compressed data corresponding to the first compression input data.

First claim

What is claimed is:

1. A data processing system comprising:
a computing platform including a hardware processor and a system memory storing a data compression software code, a trained neural encoder and a trained neural decoder, wherein the trained neural encoder and the trained neural decoder each includes parameters of a latent space probability model determined during training using a neural network;
the hardware processor configured to execute the data compression software code to:
receive a plurality of compression input data;
encode, using the trained neural encoder, a first compression input data of the plurality of compression input data to a latent space representation of the first compression input data;
decode, using the trained neural decoder, the latent space representation of the first compression input data to produce an input space representation of the first compression input data corresponding to the latent space representation of the first compression input data;
generate, using additive noise, first compression input data refined latent values based on a comparison of the first compression input data with the input space representation;
re-encode, using the trained neural encoder, the first compression input data using the first compression input data refined latent values to produce a first compressed data corresponding to the first compression input data; and
transmit the first compressed data to a remote trained neural decoder having the parameters of the latent space probability model;

wherein encoding, decoding and generating the first compression input data refined latent values do not change any of the parameters of the latent space probability model of each of the trained neural encoder and the trained neural decoder, thereby not requiring any change to any parameter of the latent space probability model of the remote trained neural decoder.

2. The data processing system of claim 1 , wherein the first compression input data refined latent values do not change any parameters of the latent space representation of the first compression input data.

3. The data processing system of claim 1 , wherein the hardware processor is further configured to execute the data compression software code to:
encode, using the trained neural encoder, a second compression input data of the plurality of compression input data to a latent space representation of the second compression input data;
decode, using the trained neural decoder, the latent space representation of the second compression input data to produce an input space representation of the second compression input data corresponding to the latent space representation of the second compression input data;
generate second compression input data refined latent values based on a comparison of the second compression input data with the input space representation of the second compression input data; and
re-encode, using the trained neural encoder, the second compression input data using the second compression input data refined latent values to produce a second compressed data corresponding to the second compression input data.

4. The data processing system of claim 1 , wherein the plurality of compression input data comprise a plurality of images.

5. The data processing system of claim 1 , wherein the plurality of compression input data comprise a video stream.

6. The data processing system of claim 1 , wherein the plurality of compression input data comprise one of a plurality of two-dimensional (2D) motion data or a plurality of color values.

7. A method for use by a data processing system including a computing platform having a hardware processor and a system memory storing a data compression software code, a trained neural encoder and a trained neural decoder, the trained neural encoder and the trained neural decoder each including parameters of a latent space probability model determined during training using a neural network, the method comprising:
receiving, by the data compression software code executed by the hardware processor, a plurality of compression input data;
encoding, using the trained neural encoder by the data compression software code executed by the hardware processor, a first compression input data of the plurality of compression input data to a latent space representation of the first compression input data;
decoding, using the trained neural decoder by the data compression software code executed by the hardware processor, the latent space representation of the first compression input data to produce an input space representation of the first compression input data corresponding to the latent space representation of the first compression input data;
generating, by the data compression software code executed by the hardware processor and using additive noise, first compression input data refined latent values, based on a comparison of the first compression input data with the input space representation;
re-encoding, using the trained neural encoder by the data compression software code executed by the hardware processor, the first compression input data using the first compression input data refined latent values to produce a first compressed data corresponding to the first compression input data;
transmitting the first compressed data to a remote trained neural decoder having the parameters of the latent space probability model;
wherein encoding, decoding and generating the first compression input data refined latent values do not change any of the parameters of the latent space probability model of each of the trained neural encoder and the trained neural decoder, thereby not requiring any change to any parameter of the latent space probability model of the remote trained neural decoder.

8. The method of claim 7 , wherein the first compression input data refined latent values do not change any parameters of the latent space representation of the first compression input data.

9. The method of claim 7 , further comprising:
encoding, using the trained neural encoder by the data compression software code executed by the hardware processor, a second compression input data of the plurality of compression input data to a latent space representation of the second compression input data;
decoding, using the trained neural decoder by the data compression software code executed by the hardware processor, the latent space representation of the second compression input data to produce an input space representation of the second compression input data corresponding to the latent space representation of the second compression input data;
generating, by the data compression software code executed by the hardware processor, second compression input data refined latent values based on a comparison of the second compression input data with the input space representation of the second compression input data; and
re-encoding, using the trained neural encoder by the data compression software code executed by the hardware processor, the second compression input data using the second compression input data refined latent values to produce a second compressed data corresponding to the second compression input data.

10. The method of claim 7 , wherein the plurality of compression input data comprise a plurality of images.

11. The method of claim 7 , wherein the plurality of compression input data comprise a video stream.

12. The method of claim 7 , wherein the plurality of compression input data comprise one of a plurality of two-dimensional (2D) motion data or a plurality of color values.

13. A data processing system comprising:
a computing platform including a hardware proc

Assignees

• Disney Entpr Inc

Inventors

• Schroers Christopher
• Meierhans Simon
• Campos Joaquim
• Mcphillen Jared
• Djelouah Abdelaziz
• Doggett Erika Varis
• Labrozzi Scott
• Xue Yuanyi

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title