Neighbor determination in video decoding

Patent metadata

Field	Value
Publication number	US-9554134-B2
Application number	US-201314100661-A
Country	US
Kind code	B2
Filing date	Dec 9, 2013
Priority date	Jun 30, 2007
Publication date	Jan 24, 2017
Grant date	Jan 24, 2017

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

First claim

We claim:

1. A method of video decoding using a video decoder, the method comprising:
with the video decoder, getting one or more tables indicating neighbor availability relationships between macroblocks, blocks and/or sub-blocks;
with the video decoder, setting up a state machine that stores macroblock neighbor availability information for plural macroblocks as plural states, wherein, for a given state among the plural states, the state machine stores information indicating a number of consecutive macroblocks in the given state and stores the macroblock neighbor availability information for the given state; and
with the video decoder, using the one or more tables and the state machine to determine neighbor availability during decoding operations.

2. The method of 1 wherein the decoding operations include one or more of context-adaptive binary arithmetic decoding, spatial intra prediction, mode computation for intra prediction, and context-adaptive variable length decoding.

3. The method of 1 wherein the using the one or more tables and the state machine involves a hierarchical determination at macroblock level then sub-macroblock level.

4. The method of 1 wherein the using the one or more tables and the state machine includes storing location information for neighboring macroblocks, and wherein the location information is stored as offsets from a current macroblock.

5. The method of 1 wherein the one or more tables include:
a first availability table that associates different index values with different macroblock neighbor availability patterns; and
a second availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

6. The method of claim 5 wherein the state machine associates the plural states with respective ones of the index values for the first availability table.

7. The method of claim 6 wherein the using the one or more tables and the state machine further comprises, during the decoding operations, for a current partition of a current macroblock:
determining a macroblock neighbor availability pattern for the current macroblock using the state machine and the first availability table; and
determining a sub-macroblock neighbor availability pattern for the current partition of the current macroblock using the macroblock neighbor availability pattern for the current macroblock and the second availability table.

8. The method of 6 wherein, for the given state, the state machine stores one of the different index values for the first availability table to indicate one of the different macroblock neighbor availability patterns for the given state.

9. The method of claim 1 wherein the one or more tables include:
an availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

10. The method of claim 9 wherein the state machine associates the plural states with the different macroblock neighbor availability patterns.

11. The method of claim 10 wherein the using the one or more tables and the state machine further comprises, during the decoding operations, for a current partition of a current macroblock:
determining a macroblock neighbor availability pattern for the current macroblock using the state machine; and
determining a sub-macroblock neighbor availability pattern for the current partition of the current macroblock using the macroblock neighbor availability pattern for the current macroblock and the availability table.

12. The method of 10 wherein, for the given state, the state machine stores one of the different macroblock neighbor availability patterns for the given state.

13. The method of claim 1 wherein the state machine is set up for a slice that includes the plural macroblocks before entropy decoding for the plural macroblocks of the slice.

14. One or more computer-readable storage media storing computer-executable instructions for causing a computer system programmed thereby to perform video decoding, wherein the one or more computer-readable storage media are selected from the group consisting of volatile memory, non-volatile memory, magnetic disk, CD-ROM, and DVD, the video decoding comprising:
getting one or more tables indicating neighbor availability relationships between macroblocks, blocks and/or sub-blocks;
setting up a state machine that stores macroblock neighbor availability information for plural macroblocks as plural states, wherein, for a given state among the plural states, the state machine stores information indicating a number of consecutive macroblocks in the given state and stores the macroblock neighbor availability information for the given state; and
using the one or more tables and the state machine to determine neighbor availability during decoding operations.

15. The one or more computer-readable storage media of claim 14 wherein the using the one or more tables and the state machine involves a hierarchical determination at macroblock level then sub-macroblock level.

16. The one or more computer-readable storage media of claim 14 wherein the one or more tables include:
a first availability table that associates different index values with different macroblock neighbor availability patterns; and
a second availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

17. A computer system including:
memory configured to store encoded data; and
a video decoder configured to decode the encoded data, thereby reconstructing video, wherein the video decoder is configured to:
get a first availability table and second availability table, wherein:
the first availability table associates different index values with different macroblock neighbor availability patterns; and
the second availability table associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns; and

set up a state machine that stores macroblock neighbor availability information for plural macroblocks in a slice as plural states, wherein, for a given state among the plural states, the state machine stores information indicating a number of consecutive macroblocks in the given state and stores one of the index values for the first availability table;
determine macroblock neighbor availability using the state machine and the first availability table; and
determine sub-macroblock neighbor availability using the macroblock neighbor availability and the second availability table.

18. A computer system that includes a processor and memory that implement a video decoder configured to perform video decoding that includes:
getting one or more tables indicating neighbor availability relationships between macroblocks, blocks and/or sub-blocks;
setting up a state machine that stores macroblock pair neighbor availability information for plural macroblock pairs as plural states, wherein, for a given state among the plural states, the state machine stores information indicating a number of consecutive macroblock pairs in the given state and stores the macroblock pair neighbor availability information for the given state; and
using the one or more tables and the state machine to determine neighbor availability during decoding operations.

19. The computer system of claim 18 wherein the using the one or

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Sermadevi Yegnaswamy
• Zhao Weidong
• Wu Yongjun

Classifications

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N19/105

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/593

  involving spatial prediction techniques · CPC title

• H04N19/159

  Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction · CPC title

• H04N19/13 Primary

  Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC] · CPC title