Multi-layer encoding and decoding

Patent metadata

Field	Value
Publication number	US-9591318-B2
Application number	US-201113235217-A
Country	US
Kind code	B2
Filing date	Sep 16, 2011
Priority date	Sep 16, 2011
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

Innovations described herein provide a generic encoding and decoding framework that includes some features of simulcast and some features of scalable video coding. For example, a bitstream multiplexer multiplexes component bitstreams into a multi-layer encoding (MLE) bitstream that provides temporal scalability, spatial resolution scalability and/or signal to noise ratio scalability. Each of the component bitstreams provides an alternative version of input video, and a given component bitstream can be a non-scalable bitstream or scalable bitstream. The multiplexer follows composition rules for the MLE bitstream and may rewrite values of certain syntax elements of component bitstreams using an approach that avoids bit shifting operations. A corresponding demultiplexer receives an MLE bitstream that includes component bitstreams and demultiplexes at least part of at least one of the component bitstreams from the MLE bitstream, following decomposition rules for the demultiplexing.

First claim

We claim:

1. In a computing system that implements a bitstream multiplexer, a method comprising:
with the computing system that implements the bitstream multiplexer, multiplexing multiple component bitstreams into a multi-layer encoding (MLE) bitstream, wherein:
each of the multiple component bitstreams provides an alternative version of input video, each of the multiple component bitstreams being independently decodable to reconstruct its version of the input video;
each of the multiple component bitstreams is a non-scalable bitstream having a single layer or a scalable bitstream having a base layer and at least one enhancement layer;
values of picture parameter set identifiers are set to avoid conflicting uses of the values of the picture parameter set identifiers between the multiple component bitstreams in the MLE bitstream, such that each of the values of the picture parameter set identifiers is unique across the multiple component bitstreams, each of the picture parameter set identifiers identifying a picture parameter set; and
the multiplexing includes inserting marker data units in the MLE bitstream, wherein each of the marker data units includes a priority identifier and one or more quality layer identifiers, the priority identifier associating a video data unit with a given layer of one of the multiple component bitstreams; and

outputting the MLE bitstream.

2. The method of claim 1 further comprising:
encoding the input video to produce the multiple component bitstreams; and
during the encoding, setting the values of the picture parameter set identifiers in the multiple component bitstreams to avoid conflicting uses of the values between the multiple component bitstreams in the MLE bitstream.

3. The method of claim 1 further comprising, with the computing system that implements the bitstream multiplexer:
receiving the multiple component bitstreams; and
rewriting one or more of the values of the picture parameter set identifiers in the multiple component bitstreams to avoid conflicting uses of the values between the multiple component bitstreams in the MLE bitstream.

4. The method of claim 3 further comprising, as part of the rewriting, for a given value of the values of the picture parameter set identifiers in the multiple component bitstreams:
identifying a set of candidate values appropriate for the given value based on number of bits in the given value;
selecting a new value from the set of candidate values; and
replacing the given value with the new value.

5. The method of claim 4 wherein the set of candidate values includes one or more candidate values having the number of bits in the given value and multiple candidate values having one or more bytes plus the number of bits in the given value, whereby the rewriting can be performed without bit shifting operations.

6. The method of claim 1 wherein at least one of the multiple component bitstreams is a scalable bitstream having a base layer and at least one enhancement layer, and wherein:
the base layer represents a first temporal resolution of the input video and a given layer of the at least one enhancement layer represents a second temporal resolution higher than the first temporal resolution; and/or
the base layer represents a first signal to noise ratio (SNR) level of the input video and a given layer of the at least one enhancement layer represents a second SNR level higher than the first SNR level.

7. The method of claim 6 wherein the alternative versions of the input video are alternative simulcast versions of the input video, such that the multiple component bitstreams provide multiple spatial resolution options between the multiple component bitstreams.

8. The method of claim 1 wherein each of the multiple component bitstreams differs from the other bitstreams of the multiple component bitstreams in terms of one or more of:
number of layers of temporal, spatial and/or signal to noise ratio (SNR) scalability, if any, supported in that component bitstream;
format of decoding of the component bitstream; and
profile and/or level for decoding of the component bitstream.

9. The method of claim 1 wherein the multiplexing enforces multiple composition rules, the multiple composition rules including:
a first composition rule according to which the priority identifiers identify layers of the multiple component bitstreams, wherein the priority identifiers are included in the marker data units for at least some slice data units;
a second composition rule according to which each of the multiple component bitstreams is assigned a non-overlapping range of the priority identifiers within which a base layer for the component bitstream has the lowest priority identifier in the range;
a third composition rule according to which the picture parameter set identifiers are each unique across the multiple component bitstreams; and
a fourth composition rule according to which sequence parameter set identifiers are each unique across the multiple component bitstreams.

10. The method of claim 1 wherein values of sequence parameter set identifiers are set to avoid conflicting uses of the values of the sequence parameter set identifiers between the multiple component bitstreams in the MLE bitstream, such that each of the values of the sequence parameter set identifiers is unique across the multiple component bitstreams, each of the sequence parameter set identifiers identifying a sequence parameter set.

11. A computing system that implements a bitstream demultiplexer, the computing system including a processor, memory and storage that stores computer-executable instructions for causing the computing system, when programmed thereby, to perform video processing comprising:
receiving a multi-layer encoding (MLE) bitstream that includes multiple component bitstreams, wherein:
each of the multiple component bitstreams is a non-scalable bitstream having a single layer or a scalable bitstream having a base layer and at least one enhancement layer;
at least one of the multiple component bitstreams is a scalable bitstream having a base layer and at least one enhancement layer that provides one or more of temporal scalability and signal to noise ratio (SNR) scalability; and
each of the multiple component bitstreams provides an alternative simulcast version of input video, such that the multiple component bitstreams provide multiple spatial resolution options between the multiple component bitstreams, each of the multiple component bitstreams being independently decodable to reconstruct its version of the input video;

demultiplexing at least part of at least one of the multiple component bitstreams from the MLE bitstream, wherein the demultiplexing is based at least in part on marker data units in the MLE bitstream, and wherein each of the marker data units includes a priority identifier and one or more quality layer identifiers, the priority identifier associating a video data unit with a given layer of one of the multiple component bitstreams; and
outputting the at least part of at least one of the multiple component bitstreams.

12. The computing system of claim 11 wherein the computing system is a multipoint conferencing unit, network server, or receiver.

13. The computing system of claim 11 wherein:
the base layer represents a first temporal resolution of the input video and a given layer of the at least one enhancement layer represents a second temporal resolution higher than the first temporal resolution; and/or
the base layer represents a first SNR level of the input video and a given layer of the at least one enhancement layer represents a second SNR level higher than

Assignees

• Lu Mei-Hsuan
• Lee Ming-Chieh
• Microsoft Technology Licensing Llc

Inventors

• Lu Mei-Hsuan
• Lee Ming-Chieh

Classifications

• H04N19/30

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N19/39 Primary

  involving multiple description coding [MDC], i.e. with separate layers being structured as independently decodable descriptions of input picture data · CPC title