Reducing memory consumption during video decoding

Patent metadata

Field	Value
Publication number	US-9819970-B2
Application number	US-201314100750-A
Country	US
Kind code	B2
Filing date	Dec 9, 2013
Priority date	Jun 30, 2007
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

First claim

We claim:

1. In a computer system that implements a video decoder, a method of video decoding comprising:
with the video decoder, receiving compressed video information in a bitstream; and
with the video decoder, decoding the compressed video information to produce one or more reconstructed pictures, wherein the decoding includes, with plural threads:
entropy decoding plural encoded transform coefficients for blocks of the one or more reconstructed pictures using the compressed video information;
packing at least some non-zero decoded transform coefficients among the decoded transform coefficients for the blocks in plural thread-specific packed buffers in memory of the computer system for intermediate storage, each of the plural thread-specific packed buffers including one or more buffer fragments, wherein the packing includes, for each individual decoded transform coefficient among the at least some non-zero decoded transform coefficients for the blocks, storing the individual decoded transform coefficient as a single multi-bit value in one of the one or more buffer fragments, the single multi-bit value including a position of the individual decoded transform coefficient within the one of the blocks and a coefficient level value of the individual decoded transform coefficient; and
dynamically adding a new buffer fragment to the one or more buffer fragments as needed to store more of the decoded transform coefficients.

2. The method of claim 1 wherein
the new buffer fragment is allocated from a buffer fragment pool that includes free buffer fragments.

3. The method of claim 1 wherein the decoding also uses an array of plural block count values for plural blocks of a macroblock, among the blocks of the one or more reconstructed pictures, each of the plural block count values indicating a count of non-zero coefficients in a corresponding block of the plural blocks of the macroblock.

4. The method of claim 1 wherein the single multi-bit value further includes an extension flag that indicates the presence or absence of a second multi-bit value for storing an extension value for the coefficient level value of the individual decoded transform coefficient.

5. The method of claim 1 wherein the packing skips explicit representation of zero-value decoded transform coefficients among the decoded transform coefficients for the blocks.

6. The method of claim 1 wherein the decoding further includes unpacking the packed transform coefficients for inverse scanning and inverse quantizing.

7. A computer system that includes a video decoder configured to perform video decoding, the computer system including:
plural processing cores configured to entropy decode plural encoded transform coefficients with plural threads;
plural thread-specific packed buffers each including one or more buffer fragments, each of the one or more buffer fragments being configured to store one or more multi-bit values for respective non-zero coefficient values among at least some of the decoded transform coefficients, each of the plural thread-specific packed buffers being configured to grow, by dynamically adding a new buffer fragment to the one or more buffer fragments, as needed to store more of the decoded transform coefficients; and
an auxiliary buffer configured to store, consecutively in an array, plural block count values for plural blocks of a macroblock, each of the plural block count values indicating a count of non-zero coefficient values among decoded transform coefficients in a corresponding block of the plural blocks.

8. The computer system of claim 7 wherein each of the one or more multi-bit values represents an individual decoded transform coefficient as a single multi-bit value, the single multi-bit value including a position of the individual decoded transform coefficient within one of the plural blocks and a coefficient level value of the individual decoded transform coefficient.

9. The computer system of claim 7 further comprising:
a buffer fragment pool that includes free buffer fragments available for dynamic addition as needed to store more of the decoded transform coefficients.

10. The computer system of claim 7 further comprising:
multiple types of memory pools available to the video decoder, each of the multiple types of memory pools including available memory chunks adapted in size for a different decoding task or different data structures used in decoding.

11. The computer system of claim 7 further comprising:
a frame buffer configured to store plural lines of a top field of a frame alternating with plural lines of a bottom field of the frame;
a frame structure configured to facilitate access to the frame in the frame buffer, wherein the frame structure includes pointers to the plural lines of the top field in the frame buffer and pointers to the plural lines of the bottom field in the frame buffer;
a top field structure configured to facilitate access to the top field in the frame buffer, wherein the top field structure includes pointers to the plural lines of the top field in the frame buffer; and
a bottom field structure configured to facilitate access to the bottom field in the frame buffer, wherein the bottom field structure includes pointers to the plural lines of the bottom field in the frame buffer.

12. The computer system of claim 7 wherein the video decoder is configured to use a graphics processing unit for the video decoding, the computer system further comprising:
a reference picture buffer configured to store one or more reference pictures as texture in memory, wherein the reference picture buffer is configured to store multiple reference pictures as a three-dimensional texture.

13. One or more computer-readable media storing computer-executable instructions for causing a computer system programmed thereby to perform video decoding operations, the one or more computer-readable media being selected from the group consisting of volatile memory, non-volatile memory, magnetic disk, CD-ROM, and DVD, the video decoding operations comprising:
receiving compressed video information; and
with plural threads, decoding the compressed video information to produce one or more reconstructed pictures for a video frame, wherein the decoding includes:
packing decoded transform coefficients into plural thread-specific packed buffers, each of the plural thread-specific packed buffers including one or more buffer fragments, each of the one or more buffer fragments storing one or more multi-bit values for respective non-zero coefficient values among the decoded transform coefficients; and
dynamically adding a new buffer fragment to the one or more buffer fragments as needed to store more of the decoded transform coefficients.

14. The one or more computer-readable media of claim 13 wherein the video frame includes a top field and a bottom field, wherein the top field includes plural lines and the bottom field includes plural lines, and wherein the video decoding operations further comprise:
buffering the top and bottom fields together in a frame buffer, the plural lines of the top field alternating with the plural lines of the bottom field in the frame buffer, wherein:
a top field structure facilitates access to the top field in the frame buffer, the top field structure including pointers to the plural lines of the top field in the frame buffer;
a bottom field structure facilitates access to the bottom field in the frame buffer, the bottom field structure including pointers to the plural lines of the bottom field in the frame buffer; and
a frame structure facilitates access to the frame in the frame buffer, the frame structur

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Sermadevi Yegnaswamy
• Zhao Weidong
• Christoffersen Eric S
• Wu Yongjun
• Arevalo Baeza Juan Carlos
• Callahan Sean

Classifications

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/82

  involving filtering within a prediction loop · CPC title

• H04N19/436

  using parallelised computational arrangements · CPC title

• H04N19/13 Primary

  Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC] · CPC title

• H04N19/124

  Quantisation · CPC title