Adaptive resolution coding for video coding

Patent metadata

Field	Value
Publication number	US-12556725-B2
Application number	US-202118259562-A
Country	US
Kind code	B2
Filing date	Dec 22, 2021
Priority date	Dec 29, 2020
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

Decoding coded pictures of a video includes receiving, by a decoder apparatus, a bitstream comprising an encoded representation of first pictures of a first resolution version of the video and decoding the coded first pictures into first pictures, the first pictures being associated with corresponding second pictures of a second resolution version of the video, the first pictures being formed by sampling the corresponding second pictures using a sampling lattice. Signalling information received in the bitstream informs the decoder apparatus about a resolution change. The method includes identifying at least one of the first pictures as a first base picture and identifying one or more auxiliary pictures, the first base picture being associated with a corresponding second picture, and, each of the one or more auxiliary pictures being formed based on sampling the second picture using a different sampling lattice of the plurality of sampling lattices. The method further includes receiving an encoded representation of the one or more auxiliary pictures in the bitstream, decoding the encoded representation of the one or more auxiliary pictures into one or more auxiliary pictures and constructing a reference picture of the second resolution based on the first base picture and the one or more auxiliary pictures. The method further includes receiving an encoded representation of second pictures in the bitstream and decoding at least part of the encoded representation of the second pictures into one or more second pictures using the constructed reference picture.

First claim

The invention claimed is:

1 . Method of decoding encoded representations of pictures of a first and second resolution version of a video by a decoder apparatus, the method including:
receiving a bitstream comprising encoded representations of first pictures of the first resolution version, each of the first pictures being a first spatially subsampled version of a corresponding second picture of the second resolution version, the subsampling being based on a first sampling lattice selected from a plurality of sampling lattices of a spatial subsampling scheme; decoding the encoded representations of the first pictures into first pictures; receiving signalling information, the signalling information informing the decoder apparatus about a termination of receiving the encoded representations of the first pictures and the start of receiving of encoded representations of second pictures of the second resolution version, the signalling information further identifying at least one of the first pictures of the first resolution version as a first base picture and selecting one or more auxiliary pictures, the first base picture being associated with a corresponding second picture of the second resolution version, and, each of the one or more auxiliary pictures being a second spatially subsampled version of the corresponding second picture of the second resolution version, the subsampling being based on one or more further sampling lattices of the plurality of sampling lattices of the spatial subsampling scheme; receiving an encoded representation of the one or more auxiliary pictures in the bitstream, decoding the encoded representation of the one or more auxiliary pictures into one or more auxiliary pictures and constructing a reference picture of the second resolution version based on the first base picture and the one or more auxiliary pictures; and, receiving an encoded representation of second pictures in the bitstream and decoding at least part of the encoded representation of the second pictures into one or more second pictures using the reference picture.

2 . Method according to claim 1 wherein the signalling information and, optionally, the one or more auxiliary pictures, is contained in one or more signalling messages, preferably one or more NAL units, such as one or more supplemental enhancement information, SEI, messages, one or more Open Bitstream Units, OBUs, or one or more messages equivalent to such SEI or OBU messages.

3 . Method according to claim 2 , wherein one or more signaling messages comprise at least one identifier, for example a frame number, identifying the first base picture in the sequence of coded first pictures.

4 . Method according to claim 1 wherein the first pictures have a first resolution and the second pictures have a second resolution, the signaling information including: information about the second resolution and/or information about the identity of one or more VCL NAL units comprising a coded representation of the one or more auxiliary pictures.

5 . Method according to claim 1 wherein the signaling information further comprises information for constructing the reference picture, preferably information associated with the sampling lattices that were used to form the base picture and the one or more auxiliary pictures and/or information associated with an interpolation filter for interpolating pixels values for the reference picture.

6 . Method according to claim 1 wherein the constructing of the reference picture includes:
upscaling the base picture and the one or more auxiliary pictures to an upscaled base picture and one or more upscaled auxiliary picture respectively, the upscaling being based on the sampling lattices of the spatial subsampling scheme;
summing the upscaled base picture and the one or more upscaled auxiliary pictures into a summed picture.

7 . Method according to claim 1 , wherein the encoded representation of the one or more auxiliary pictures is computed using the base picture as a reference frame for at least one of the one or more auxiliary pictures.

8 . Method according to claim 1 , wherein a sampling lattice defining pixels of a second picture is selected to form the first picture.

9 . Method according to claim 1 wherein the decoding is based on a coding standard, preferably a block-based video coding standard, more preferably an AVC, HEVC, VP9, AV1, VVC coding standard or a coding standard based on of the AVC, HEVC, VP9, AV1, VVC coding standards.

10 . Method of transmitting a bitstream comprising encoded representations of pictures of a first and second resolution version of a video to a device, the method comprising:
transmitting a bitstream to the device, the bitstream comprising an encoded representation of first pictures of the first resolution version, the device comprising a decoder apparatus for decoding the encoded representation of first pictures into first pictures, the first pictures being associated with corresponding second pictures of the second resolution version of the video, each of the first pictures being a first spatially subsampled version of a corresponding second picture of the second resolution version, the subsampling being based on a first sampling lattice selected from a plurality of sampling lattices of a spatial subsampling scheme; transmitting signalling information to the device, the signalling information informing the decoder apparatus about a termination of receiving the encoded representations of the first pictures and the start of receiving of encoded representations of second pictures of the second resolution version, the signalling information further identifying at least one of the first pictures of the first resolution version as a first base picture and identifying one or more auxiliary pictures, the first base picture being associated with a corresponding second picture of the second resolution version, and, each of the one or more auxiliary pictures being a second spatially subsampled version of the corresponding second picture of the second resolution version, the subsampling being based on one or more further sampling lattices of the plurality of sampling lattices of the spatial subsampling scheme; transmitting an encoded representation of the one or more auxiliary pictures in the bitstream to the device for the decoder apparatus to decode the encoded representation of the one or more auxiliary pictures into one or more auxiliary pictures and construct a reference picture of the second resolution version based on the first base picture and the one or more auxiliary pictures; and, transmitting an encoded representation of second pictures in the bitstream to the device for decoding at least part of the encoded representation of the second pictures into one or more second pictures based on the reference picture.

11 . A decoder apparatus for decoding encoded representations of pictures of a first and second resolution version of a video, the decoder apparatus comprising:
a computer readable storage medium having computer readable program code embodied therewith, and a processor coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising: receiving a bitstream comprising encoded representations of first pictures of the first resolution version, each of the first pictures being a first spatially subsampled version of a corresponding second picture of the second resolution version, the subsampling being based on a first sampling lattice selected from a plurality of sampling lattices of a spatial subsampling scheme; decoding the encoded representations

Assignees

• Koninklijke Kpn Nv
• TNO

Inventors

• Thomas Emmanuel
• Da Silva Pratas Gabriel Alexandre
• Potetsianakis Emmanouil

Classifications

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N19/188

  the unit being a video data packet, e.g. a network abstraction layer [NAL] unit · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title