Block-based long-range context model in neural image compression

Patent metadata

Field	Value
Publication number	US-12495151-B2
Application number	US-202318458507-A
Country	US
Kind code	B2
Filing date	Aug 30, 2023
Priority date	Dec 27, 2022
Publication date	Dec 9, 2025
Grant date	Dec 9, 2025

Abstract

Methods and apparatuses for decoding a compressed image using a neural image compression network may be provided. The method may include generating long-range context model parameters associated with a high resolution compressed image, the long-range context model parameters corresponding to a first area. The method may also include splitting the generated long-range context model parameters into a first number of context parameter blocks. The method may also include for each block in the first number of context parameter blocks, predicting respective context features using a long-range context model and respective context parameter blocks, wherein the long-range context model uses a corner-to-center latent decoding strategy or an edge-to-center latent decoding strategy to decode latents associated with the high resolution compressed image. Then, the high resolution compressed image may be reconstructed based on predicted context features.

First claim

What is claimed is:

1 . A method for decoding a compressed image using a neural image compression network, the method being executed by at least one processor, the method comprising:
receiving a high resolution compressed image; generating long-range context model parameters associated with the high resolution compressed image, the long-range context model parameters corresponding to a first area; splitting the generated long-range context model parameters into a first number of context parameter blocks; for each block in the first number of context parameter blocks, predicting respective context features using a long-range context model and respective context parameter blocks, wherein the long-range context model uses a corner-to-center latent decoding or an edge-to-center latent decoding to decode latents associated with the high resolution compressed image; and reconstructing the high resolution compressed image based on the predicted context features.

2 . The method of claim 1 , wherein the predicting the respective context features of each of the first number of context parameter blocks is performed in parallel.

3 . The method of claim 1 , wherein the predicting the respective context features of each of the first number of context parameter blocks is performed in series.

4 . The method of claim 1 , wherein predicting the respective context features comprises:
predicting first context features using the long-range context model and first long-range context model parameters from a first context parameter block,
wherein the first long-range context model parameters correspond to first locations in a first split area, and
wherein the first context features that are predicted using the first long-range context model parameters correspond to second locations in the first split area; and
predicting second context features using the long-range context model and second long-range context model parameters from the first context parameter block,
wherein the second long-range context model parameters correspond to the second locations in the first split area, and
wherein the second context features that are predicted using the second long-range context model parameters correspond to third locations in the first split area.

5 . The method of claim 4 , wherein according to the corner-to-center latent decoding comprises:
the first locations in the first split area are corner locations in the first split area, the second locations in the first split area are locations at mid-points between one or more of the corner locations, and the third locations in the first split area are locations at mid-points between one or more of the second locations.

6 . The method of claim 4 , wherein the edge-to-center latent decoding comprises:
the first locations in the first split area being edge locations in the first split area, the second locations in the first split area being locations at mid-points between one or more of the edge locations or corner locations associated respective edges of the first split area, and the third locations in the first split area being locations at mid-points between one or more of the second locations.

7 . The method of claim 1 , wherein each of the first number of split areas has a different shape, and wherein each of the first number of split areas is one of a square area or a rectangle area.

8 . The method of claim 1 , wherein the long-range context model is a transformer-based context prediction model.

9 . An apparatus for decoding a compressed image using a neural image compression network, the apparatus comprising:
at least one memory configured to store computer program code; and at least one processor configured to read the computer program code and operate as instructed by the computer program code, the computer program code including:
receiving code configured to cause the at least one processor to receive a high resolution compressed image;
generating code configured to cause the at least one processor to generate long-range context model parameters associated with the high resolution compressed image, the long-range context model parameters corresponding to a first area;
splitting code configured to cause the at least one processor to split the generated long-range context model parameters into a first number of context parameter blocks;
predicting code configured to cause the at least one processor to predict, for each block in the first number of context parameter blocks, respective context features using a long-range context model and respective context parameter blocks, wherein the long-range context model uses a corner-to-center latent decoding or an edge-to-center latent decoding to decode latents associated with the high resolution compressed image; and
reconstructing code configured to cause the at least one processor to reconstruct the high resolution compressed image based on the predicted context features.

10 . The apparatus of claim 9 , wherein the predicting the respective context features of each of the first number of context parameter blocks is performed in parallel.

11 . The apparatus of claim 9 , wherein the predicting the respective context features of each of the first number of context parameter blocks is performed in series.

12 . The apparatus of claim 9 , wherein the predicting code is further configured to cause the at least one processor to:
predict first context features using the long-range context model and first long-range context model parameters from a first context parameter block,
wherein the first long-range context model parameters correspond to first locations in a first split area, and
wherein the first context features that are predicted using the first long-range context model parameters correspond to second locations in the first split area; and
predict second context features using the long-range context model and second long-range context model parameters from the first context parameter block,
wherein the second long-range context model parameters correspond to the second locations in the first split area, and
wherein the second context features that are predicted using the second long-range context model parameters correspond to third locations in the first split area.

13 . The apparatus of claim 12 , wherein according to the corner-to-center latent decoding comprises:
the first locations in the first split area are corner locations in the first split area, the second locations in the first split area are locations at mid-points between one or more of the corner locations, and the third locations in the first split area are locations at mid-points between one or more of the second locations.

14 . The apparatus of claim 9 , wherein each of the first number of split areas has a different shape, and wherein each of the first number of split areas is one of a square area or a rectangle area.

15 . A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of an apparatus for decoding a compressed image using a neural image compression network, cause the at least one processor to:
receive a high resolution compressed image; generate long-range context model parameters associated with the high resolution compressed image, the long-range context model parameters corresponding to a first area; split the generated long-range context model parameters into a first number of context parameter blocks; for each block in the first number of context parameter blocks, predict respective context features using a long-range context model and r

Assignees

• Tencent America LLC

Inventors

• Sui Yang
• Ding Ding
• Xu Xiaozhong
• Liu Shan

Classifications

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/119

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title