Partial output of a decoded picture buffer in video coding

Patent metadata

Field	Value
Publication number	US-12401811-B2
Application number	US-202118259559-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Dec 28, 2020
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

Methods and apparatus for processing an encoded representation of a sequence of pictures are disclosed, wherein the method includes: a processor of a decoding apparatus allocating memory for one or more pictures in a decoded picture buffer, DPB; the processor receiving a bitstream comprising an encoded representation of a current picture, the encoded representation of the current picture comprising a plurality of decoding units, DUs, each of the plurality of DUs comprising an encoded set of pixels; the processor storing the plurality of DUs of the current picture in a coded picture buffer, CPB; the processor decoding the plurality of DUs into decoded DUs of the current picture and storing the decoded DUs in the allocated memory of the DPB, the processor executing at least one partial output during the decoding of the plurality of DUs of the current picture, the performing of the at least one partial output including: copying one or more of the decoded DUs from the DPB to a data sink before all of the plurality of DUs are decoded, the one or more decoded DUs representing one or more areas of pixels in the current picture; determining output information comprising information about the one or more areas of pixels that are copied to the data sink and information for identifying the current picture; providing access to the output information for use by an application or an application programming interface, API, outside the decoding apparatus.

First claim

The invention claimed is:

1. A method of processing an encoded representation of a sequence of pictures, preferably the sequence of pictures defined by coded pictures that are independently decodable from coded pictures outside the sequence, the method including:
allocating, by a processor of a decoding apparatus, memory for one or more pictures in a decoded picture buffer, DPB;
receiving, by the processor, a bitstream comprising an encoded representation of a current picture, the encoded representation of the current picture comprising a plurality of decoding units, DUs, each of the plurality of DUs comprising an encoded set of pixels;
storing, by the processor, the plurality of DUs of the current picture in a coded picture buffer, CPB;
decoding, by the processor, the plurality of DUs into decoded DUs of the current picture and storing the decoded DUs in the allocated memory of the DPB;
executing, by the processor, at least one partial output during the decoding of the plurality of DUs of the current picture, the executing of the at least one partial output including:
copying one or more of the decoded DUs from the DPB to a data sink before all of the plurality of DUs are decoded, each decoded DU comprising a set of pixels representing an area in the current picture, the data sink being accessible by an application outside the decoder apparatus.

2. The method according to claim 1 , wherein the executing of the at least one partial output further includes:
determining output information identifying areas in the current picture associated with the one or more copied decoded DUs; and, optionally,
providing access to the output information for use by the application, preferably via an application programming interface, API.

3. The method according to claim 2 wherein the output information further comprises at least one of:
information indicative of whether the encoded representation of the current picture is fully decoded or not;
information indicative of the number DUs that are not yet decoded and/or the number of DUs that are decoded; or, information for determining the number of DUs that are not yet decoded and/or the number of DUs that are decoded;
one or more areas in the current picture that are not yet decoded and/or one or more areas in the current picture that are decoded.

4. The method according to claim 1 , wherein the providing access to the output information includes:
signaling the application or the API that the partial output has been performed, preferably the signaling being executed in response to the copying of the one or more of the decoded DUs to the data sink; or, in response to receiving a request from the application or the API.

5. The method according to claim 1 wherein the sequence of pictures is associated with an output order, the output order defining the order wherein the pictures in the sequence of pictures are output by the decoder apparatus, preferably the output order being based on picture order information, such as picture order count, POC, values, the picture order information identifying a picture in the sequence of pictures and being indicative of an output order position of the picture relative to the output order position of the other pictures in the sequence, wherein the copying the one or more decoded DUs to the data sink is executed, if the output order signals that the current picture can be output; preferably the POC value of the current picture signaling that the current picture can be output.

6. The method according to claim 1 wherein the sequence of pictures is associated with an output order, the output order defining the order wherein the pictures in the sequence of pictures are output by the decoder apparatus, each picture of the sequence of pictures being associated with an output order position, the sequence of pictures further being associated with a decoding order, the decoding order defining the order wherein the pictures in the sequence of pictures are decoded by the decoder apparatus, each picture of the sequence of pictures being associated with a decoding order position;
wherein the determining output information further comprises:
adding an output order position identifier to the output information, if the decoding order position of the current picture is different from the output order position of the current picture, preferably the output order position identifier being based on picture order information, for example one or more picture order count, POC, values, a POC value identifying a picture in the sequence of pictures and being indicative of an output order position of the picture relative to the output order position of other pictures in the sequence.

7. The method according to claim 1 , wherein the performing of at least one partial output further includes:
marking the one or more decoded DUs stored in the DPB as being ready for partial output, the marking signaling the decoder apparatus not to remove the one or more decoded DUs from the DPB in response to the performance of the at least one partial output.

8. The method according to claim 1 wherein before performing the at least one partial output, the processor determining if a partial output can or should be performed, the determining including at least one of:
determining if metadata, preferably a partial output flag and/or a signaling message, such as a supplemental enhancement information, SEI, message for example a picture timing SEI message, in the bitstream signals that a partial output can or should be performed;
determining if metadata, preferably a filtering flag, in the bitstream signals that inloop filtering is activated or deactivated, preferably that inloop filtering is activated or deactivated across a boundary between at least two neighbouring decoded DUs stored in the DPB.

9. The method according to claim 8 , wherein if the metadata signals that partial output can or should be done, determining if the metadata comprises timing information for performing one or more partial outputs during the decoding of the plurality of DUs, preferably the timing information including an output interval for performing the one or more partial outputs.

10. The method according to claim 1
wherein the data sink includes at least one buffer that at least matches the size of a decoded DU, the performing of at least one partial output including: copying pixels of the decoded DU in the DPB to the at least one buffer;
or,
wherein the data sink comprises at least n buffers, n being an integer number larger than two, preferably n being equal to the number of the plurality of DUs, the size of each of the n buffers at least matching the size of a decoded DU, the performing of at least one partial output including: copying pixels of the decoded DU in at least one of the n buffers;
or,
wherein the data sink includes at least one buffer that at least matches the size of at least one picture or an integer number of pictures, the performing of at least one partial output including: copying pixels of the DU into the buffer at a position according to the position of the pixels of decoded DU in the current picture.

11. The method according to claim 1
wherein a DU is based on one or more macroblocks as defined in AVC, one or more coded tree units CTUs as defined in HEVC, one or more slices as defined in HEVC or VVC or one or more sub-pictures as defined in VVC; and/or,
wherein the decoding is based on a coding standard, preferably a block-based video coding standard, more preferably an AVC, HEVC, VP9, AV1, VVC, EVC coding standard or a coding standard based on of the AVC, HEVC, VP9, AV1, VVC, EVC coding standard.

12. A decoding apparatus for processi

Assignees

• Koninklijke Kpn Nv
• TNO

Inventors

• Potetsianakis Emmanouil
• Thomas Emmanuel

Classifications

• H04N19/182

  the unit being a pixel · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N19/136

  Incoming video signal characteristics or properties · CPC title

• G06F9/547

  Remote procedure calls [RPC]; Web services · CPC title

• H04N19/423

  characterised by memory arrangements (H04N19/433 takes precedence) · CPC title