Hardware-accelerated decoding of scalable video bitstreams

Patent metadata

Field	Value
Publication number	US-9819949-B2
Application number	US-201113329093-A
Country	US
Kind code	B2
Filing date	Dec 16, 2011
Priority date	Dec 16, 2011
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

In various respects, hardware-accelerated decoding is adapted for decoding of video that has been encoded using scalable video coding. For example, for a given picture to be decoded, a host decoder determines whether a corresponding base picture will be stored for use as a reference picture. If so, the host decoder directs decoding with an accelerator such that the some of the same decoding operations can be used for the given picture and the reference base picture. Or, as another example, the host decoder groups encoded data associated with a given layer representation in buffers. The host decoder provides the encoded data for the layer to the accelerator. The host decoder repeats the process layer-after-layer in the order that layers appear in the bitstream, according to a defined call pattern for an acceleration interface, which helps the accelerator determine the layers with which buffers are associated.

First claim

We claim:

1. A tangible computer-readable medium storing computer-executable instructions for causing a computing system, when programmed thereby, to perform scalable video decoding control comprising:
receiving at least part of a bitstream for video data having been encoded using scalable video coding, the bitstream including encoded data for a given picture to be decoded for output, the given picture having a reference base picture to be decoded and stored for use as a reference picture but not output; and
with a host decoder, calling an acceleration interface to direct decoding of the given picture and decoding of the reference base picture by an accelerator, including interleaving, according to a call pattern, at least some calls for the decoding of the reference base picture with at least some calls for the decoding of the given picture, wherein, according to the call pattern, the calling the acceleration interface includes:
before calling a second routine for the given picture:
calling a first routine to signal initiation of the decoding of the reference base picture; and
calling the first routine to signal initiation of the decoding of the given picture;

calling the second routine one or more times to signal that at least some of the encoded data for the given picture is accessible to the accelerator; and
after the calling the second routine for the given picture:
calling a third routine to signal that encoded data to be used in the decoding of the reference base picture has been provided; and
calling the third routine to signal that encoded data to be used in the decoding of the given picture has been provided.

2. The computer-readable medium of claim 1 wherein the interleaving facilitates recognition by the accelerator of opportunities to share operations between the decoding of the reference base picture and the decoding of the given picture.

3. The computer-readable medium of claim 1 wherein, for the reference base picture, the calling the first routine and the calling the third routine include a first index to a first output surface associated with the reference base picture, and wherein, for the given picture, the calling the first routine and the calling the third routine include a second index to a second output surface associated with the given picture.

4. The computer-readable medium of claim 1 wherein the bitstream further includes encoded data for a next picture to be decoded for output, the next picture lacking a base picture to be stored for use as a reference picture, the scalable video decoding control further comprising:
calling the first routine to signal initiation of the decoding of the next picture;
calling the second routine one or more times to signal that the encoded data for the next picture is accessible to the accelerator; and
calling the third routine to signal that encoded data to be used in the decoding of the next picture has been provided.

5. The computer-readable medium of claim 1 wherein the scalable video decoding control further comprises:
deciding whether to initiate concurrent decoding of the given picture and the reference base picture;
if so, performing the calling the acceleration interface to direct the decoding of the given picture and the decoding of the reference base picture by the accelerator; and
otherwise, calling the acceleration interface to direct the decoding of the given picture by the accelerator.

6. The computer-readable medium of claim 1 wherein the encoded data for the given picture has multiple layers in coded order in the bitstream, and wherein the calling the second routine one or more times includes, in the coded order, for each of the multiple layers:
storing encoded data for that layer in multiple buffers accessible to the accelerator; and
calling the second routine to signal that the encoded data for that layer is accessible.

7. The computer-readable medium of claim 6 wherein the multiple layers include a target layer associated with the given picture and a reference layer associated with the reference base picture.

8. The computer-readable medium of claim 6 wherein the multiple buffers include a picture parameters buffer, a slice control buffer and one or more bitstream data buffers.

9. The computer-readable medium of claim 1 wherein the bitstream includes units of the encoded data for the given picture, wherein the encoded data for the given picture has multiple layers in coded order in the bitstream, and wherein the scalable video decoding control further comprises:
with the host decoder, detecting loss of units of encoded data used for reconstruction of a target layer of the multiple layers; and
with the host decoder, changing the target layer for at least some of the units of the encoded data for the given picture, wherein the units of the encoded data with the changed target layer are provided to the accelerator.

10. The computer-readable medium of claim 9 wherein the units of the encoded data for the given picture are units of slice data, and wherein the lost units are for the target layer and/or for reference in inter-layer prediction.

11. The computer-readable medium of claim 9 wherein the changing comprises altering layer type syntax elements in the units of the encoded data for the given picture.

12. The computer-readable medium of claim 1 wherein the call pattern signals to the accelerator to avoid redundant performance of at least some operations between the decoding of the given picture and the decoding of the reference base picture.

13. A method comprising:
receiving at least part of a bitstream for video data having been encoded using scalable video coding, the bitstream including encoded data for a given picture to be decoded for output, the given picture having a reference base picture to be decoded and stored for use as a reference picture but not output; and
with a host decoder, calling an acceleration interface to direct decoding of the given picture and decoding of the reference base picture by an accelerator, including interleaving, according to a call pattern, at least some calls for the decoding of the reference base picture with at least some calls for the decoding of the given picture, wherein, according to the call pattern, the calling the acceleration interface includes:
before calling a second routine for the given picture:
calling a first routine to signal initiation of the decoding of the reference base picture; and
calling the first routine to signal initiation of the decoding of the given picture;

calling the second routine one or more times to signal that at least some of the encoded data for the given picture is accessible to the accelerator; and
after the calling the second routine for the given picture:
calling a third routine to signal that encoded data to be used in the decoding of the reference base picture has been provided; and
calling the third routine to signal that encoded data to be used in the decoding of the given picture has been provided.

14. The method of claim 13 wherein, for the reference base picture, the calling the first routine and the calling the third routine include a first index to a first output surface associated with the reference base picture, and wherein, for the given picture, the calling the first routine and the calling the third routine include a second index to a second output surface associated with the given picture.

15. The method of claim 13 wherein the method further comprises:
deciding whether to initiate concurrent decoding of the given picture and the reference base picture;
if so, performing the calling the acceler

Assignees

• Lu Mei-Hsuan
• Wu Yongjun
• Lee Ming-Chieh
• Sullivan Gary J
• Microsoft Technology Licensing Llc

Inventors

• Lu Mei-Hsuan
• Wu Yongjun
• Lee Ming-Chieh
• Sullivan Gary J

Classifications

• H04N19/30

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N19/423

  characterised by memory arrangements (H04N19/433 takes precedence) · CPC title

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N19/436

  using parallelised computational arrangements · CPC title