Temporal resampling and restoration in video coding and decoding systems

Patent metadata

Field	Value
Publication number	US-2025039387-A1
Application number	US-202418676147-A
Country	US
Kind code	A1
Filing date	May 28, 2024
Priority date	Jul 30, 2023
Publication date	Jan 30, 2025
Grant date	—

Abstract

This disclosure relates generally to video coding/decoding and particularly for signaling in temporal resampling and restoration in video coding and/or decoding systems. One method includes receiving the coded video bitstream; extracting, from the coded video bitstream, a syntax indicating a temporal restoration mode, wherein the syntax comprises: a flag and an index, the flag indicating whether temporal restoration is enabled, the index indicating one temporal resampling ratio out of 2{circumflex over ( )}N non-unity temporal resampling ratios, wherein the index has N bits, or an index only, the index indicating whether the temporal restoration is enabled and one temporal resampling ratio of at least one temporal resampling ratios; and when the temporal restoration is enabled, restoring, by the device, reconstructed frames to obtain restored frames based on the indicated temporal resampling ratio.

First claim

What is claimed is:

1 . A method for decoding a coded video bitstream, the method comprising:
receiving, by a device comprising a memory storing instructions and a processor in communication with the memory, a coded video bitstream; extracting, by the device from the coded video bitstream, a syntax indicating a temporal restoration mode, wherein the syntax comprises:
a flag and an index, the flag indicating whether temporal restoration is enabled, the index indicating one temporal resampling ratio out of 2{circumflex over ( )}N non-unity temporal resampling ratios, wherein the index has N bits, or
an index only, the index indicating whether the temporal restoration is enabled and one temporal resampling ratio of at least one temporal resampling ratios; and
when the temporal restoration is enabled, restoring, by the device, reconstructed frames to obtain restored frames based on the indicated temporal resampling ratio.

2 . The method according to claim 1 , wherein the restoring reconstructed frames to obtain restored frames based on the indicated temporal resampling ratio comprises:
performing, based on the indicated temporal resampling ratio, frame interpolation based on the reconstructed frames to generate frames to be added with the reconstructed frames to obtain the restored frames.

3 . The method according to claim 1 , wherein, when the syntax comprises the flag and the index:
when the flag is “1”, the flag indicates that the temporal restoration is enabled; and when the flag is “0”, the flag indicates that the temporal restoration is disabled.

4 . The method according to claim 1 , wherein, when the syntax comprises the flag and the index:
the temporal resampling ratio is indicated by the index by:
being equal to (M+2) or 2{circumflex over ( )}(M+1), wherein M is an unsigned integer value of the index.

5 . The method according to claim 1 , wherein, when the syntax comprises the flag and the index:
the temporal resampling ratio is indicated by the index by:
the index indicating an entry value in a look-up table, wherein the look-up table comprises 2{circumflex over ( )}N non-unity temporal resampling ratios from 2 to (2{circumflex over ( )}N+1) or from 2 to 2{circumflex over ( )}(N+1).

6 . The method according to claim 1 , wherein, when the syntax comprises the index only:
when the index is zero, the temporal restoration is disabled; when the index is larger than zero, the temporal restoration is enabled, and the temporal resampling ratio is (M+1) or 2{circumflex over ( )}(M+1), wherein M is an unsigned integer value of the index.

7 . The method according to claim 1 , wherein, when the syntax comprises the index only:
the index indicates an entry value in a look-up table, wherein the look-up tables comprises values from 1 to (2{circumflex over ( )}N−1) or from 1 to 2{circumflex over ( )}(M−1); when the entry value is “1”, the temporal restoration is disabled; when the entry value is larger than 1, the temporal restoration is enabled, and the temporal resampling ratio is the entry value, wherein M is an unsigned integer value of the index.

8 . An apparatus for decoding a current block of a current frame in a coded video bitstream, the apparatus comprising:
a memory storing instructions; and a processor in communication with the memory, wherein, when the processor executes the instructions, the processor is configured to cause the apparatus to:
receive a coded video bitstream;
extract, from the coded video bitstream, a syntax indicating a temporal restoration mode, wherein the syntax comprises:
a flag and an index, the flag indicating whether temporal restoration is enabled, the index indicating one temporal resampling ratio out of 2{circumflex over ( )}N non-unity temporal resampling ratios, wherein the index has N bits, or
an index only, the index indicating whether the temporal restoration is enabled and one temporal resampling ratio of at least one temporal resampling ratios; and

when the temporal restoration is enabled, restore reconstructed frames to obtain restored frames based on the indicated temporal resampling ratio.

9 . The apparatus according to claim 8 , wherein, when the processor is configured to cause the apparatus to restore reconstructed frames to obtain the restored frames based on the indicated temporal resampling ratio, the processor is configured to cause the apparatus to:
perform, based on the indicated temporal resampling ratio, frame interpolation based on the reconstructed frames to generate frames to be added with the reconstructed frames to obtain the restored frames.

10 . The apparatus according to claim 8 , wherein, when the syntax comprises the flag and the index:
when the flag is “1”, the flag indicates that the temporal restoration is enabled; and when the flag is “0”, the flag indicates that the temporal restoration is disabled.

11 . The apparatus according to claim 8 , wherein, when the syntax comprises the flag and the index:
the temporal resampling ratio is indicated by the index by:
being equal to (M+2) or 2{circumflex over ( )}(M+1), wherein M is an unsigned integer value of the index.

12 . The apparatus according to claim 8 , wherein, when the syntax comprises the flag and the index:
the temporal resampling ratio is indicated by the index by:
the index indicating an entry value in a look-up table, wherein the look-up table comprises 2{circumflex over ( )}N non-unity temporal resampling ratios from 2 to (2{circumflex over ( )}N+1) or from 2 to 2{circumflex over ( )}(N+1).

13 . The apparatus according to claim 8 , wherein, when the syntax comprises the index only:
when the index is zero, the temporal restoration is disabled; when the index is larger than zero, the temporal restoration is enabled, and the temporal resampling ratio is (M+1) or 2{circumflex over ( )}(M+1), wherein M is an unsigned integer value of the index.

14 . The apparatus according to claim 8 , wherein, when the syntax comprises the index only:
the index indicates an entry value in a look-up table, wherein the look-up tables comprises values from 1 to (2{circumflex over ( )}N−1) or from 1 to 2{circumflex over ( )}(M−1); when the entry value is “1”, the temporal restoration is disabled; when the entry value is larger than 1, the temporal restoration is enabled, and the temporal resampling ratio is the entry value, wherein M is an unsigned integer value of the index.

15 . A non-transitory computer readable storage medium storing instructions, wherein, when the instructions are executed by a processor, the instructions are configured to cause the processor to:
receive a coded video bitstream; extract, from the coded video bitstream, a syntax indicating a temporal restoration mode, wherein the syntax comprises:
a flag and an index, the flag indicating whether temporal restoration is enabled, the index indicating one temporal resampling ratio out of 2{circumflex over ( )}N non-unity temporal resampling ratios, wherein the index has N bits, or
an index only, the index indicating whether the temporal restoration is enabled and one temporal resampling ratio of at least one temporal resampling ratios; and
when the temporal restoration is enabled, restore reconstructed frames to obtain restored frames based on the indicated temporal resampling ratio.

16 . The non-transitory computer readable storage medium according to claim 15 , wherein, when the syntax comprises the flag and the index:
when the flag is “1”, the flag indicates that the temporal restoration is enabled; and when the flag i

Assignees

• Tencent America LLC

Inventors

• Liu Shan
• Ding Ding

Classifications

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/31

  in the temporal domain · CPC title

• H04N19/59

  involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution · CPC title

• H04N19/132 Primary

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title