Non-nested SEI messages in video coding

Patent metadata

Field	Value
Publication number	US-9521393-B2
Application number	US-201314036873-A
Country	US
Kind code	B2
Filing date	Sep 25, 2013
Priority date	Jan 7, 2013
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

A device obtains, from a bitstream that includes an encoded representation of the video data, a non-nested Supplemental Enhancement Information (SEI) message that is not nested within another SEI message in the bitstream. Furthermore, the device determines a layer of the bitstream to which the non-nested SEI message is applicable. The non-nested SEI message is applicable to layers for which video coding layer (VCL) network abstraction layer (NAL) units of the bitstream have layer identifiers equal to a layer identifier of a SEI NAL unit that encapsulates the non-nested SEI message. A temporal identifier of the SEI NAL unit is equal to a temporal identifier of an access unit containing the SEI NAL unit. Furthermore, the device processes, based in part on one or more syntax elements in the non-nested SEI message, video data of the layer of the bitstream to which the non-nested SEI message is applicable.

First claim

What is claimed is:

1. A method of processing video data, the method comprising:
obtaining, from a bitstream of coded video data that includes a sequence of network abstraction layer (NAL) units, a non-nested Supplemental Enhancement Information (SEI) message, wherein the non-nested SEI message is not nested within a scalable nesting SEI message in the bitstream and the non-nested SEI message is not a buffering period SEI message, a picture timing SEI message, or a decoding unit information SEI message;
determining a layer of the bitstream to which the non-nested SEI message is applicable, wherein the layer to which the non-nested SEI message is applicable is a layer for which video coding layer (VCL) NAL units of the bitstream have layer identifiers equal to a layer identifier specified in a SEI NAL unit that encapsulates the non-nested SEI message, and wherein a temporal identifier specified in the SEI NAL unit is required to be equal to a temporal identifier of an access unit containing the SEI NAL unit; and
processing, based in part on data in the non-nested SEI message, video data of the layer of the bitstream to which the non-nested SEI message is applicable.

2. The method of claim 1 , wherein:
the SEI NAL unit includes a NAL unit header and a payload,
the NAL unit header of the SEI NAL unit includes at least a first syntax element and a second syntax element,
the first syntax element specifies the layer identifier specified in the SEI NAL unit,
the second syntax element specifies the temporal identifier specified in the SEI NAL unit, and
the payload of the SEI NAL unit includes the non-nested SEI message.

3. The method of claim 1 , wherein the non-nested SEI message is one of:
a pan-scan rectangle SEI message that includes data associated with displaying at a different picture aspect ratio than a picture aspect ratio of output pictures,
a filler payload SEI message that includes data for adjusting a bit rate to meet specific constraints,
a recovery point SEI message that includes information for clean random access or gradual decoding refresh,
a scene information SEI message that includes information associated with scene changes and transitions,
a picture snapshot SEI message that includes an indication to label an associated decoded picture as a still-image snapshot of video content,
a progressive refinement segment start SEI message that includes information associated with a start of a segment of consecutive pictures that represent a progressive refinement of quality of a picture rather than a moving scene,
a progressive refinement segment end SEI message that includes information associated with an end of the segment of consecutive pictures,
a film grain characteristics SEI message that includes information associated with synthesizing film grain effects,
a post filter hint SEI message that includes information associated with suggested post-filter coefficients or correlation information for post-filter design,
a tone mapping information SEI message that includes information associated with remapping to another color space than that used or assumed in encoding,
a frame packing arrangement SEI message that includes information associated with packing of stereoscopic video into the bitstream,
a display orientation SEI message that includes information that specifies flipping and/or rotation to be applied to the output pictures when the output pictures are displayed,
a structure of pictures information SEI message that includes information that describes temporal and inter prediction structure of the bitstream,
a temporal sub-layer zero index SEI message that includes information associated with detecting when coded pictures with temporal identifiers equal to 0 are missing, or
a region refresh information SEI message that includes information that indicates whether slice segments to which the non-nested SEI message applies belong to a refreshed region of a current picture.

4. The method of claim 3 , wherein the method further comprises determining, based on one or more syntax elements of the non-nested SEI message, a payload type variable for the non-nested SEI message, wherein:
the payload type variable for the non-nested SEI message indicates a payload type of the non-nested SEI message, and
the payload type variable for the non-nested SEI message is equal to 2, 3, 6, 9, 15, 16, 17, 19, 22, 23, 45, 47, 128, 131, or 134.

5. The method of claim 1 , wherein:
the non-nested SEI message is a first non-nested SEI message, and
the method further comprises:
obtaining, from the bitstream, a second non-nested buffering period SEI message that indicates initial delays for hypothetical reference decoder (HRD) operations;
when the second non-nested buffering period SEI message is applicable to TargetOp, selecting, as an HRD initialization point, an access unit associated with the second non-nested buffering period SEI message, wherein TargetOp is the set of layer identifiers in a bitstream-to-decode, the bitstream-to-decode being the bitstream or a sub-stream of the bitstream; and
initializing a HRD at the selected access unit.

6. The method of claim 1 , wherein:
the non-nested SEI message is a first non-nested SEI message, and
the method further comprises:
obtaining, from the bitstream, a second non-nested SEI message, the second non-nested SEI message being a buffering period SEI message that indicates initial delays for HRD operations, a picture timing SEI message that indicates picture output times and picture/sub-picture removal times for HRD operations, or a decoding unit SEI message that indicates sub-picture removal times for HRD operations; and
for each respective access unit in a bitstream-to-decode:
selecting the second non-nested SEI message for the respective access unit when the second non-nested SEI message is associated with the access unit and the second non-nested SEI message is applicable to the set of layer identifiers in the bitstream-to-decode, wherein the bitstream-to-decode is the bitstream or a sub-bitstream of the bitstream; and
using one or more syntax elements of the second non-nested SEI message in an HRD operation with regard to the respective access unit.

7. The method of claim 1 , wherein:
the non-nested SEI message is a first non-nested SEI message, and
the method further comprises:
obtaining, from the bitstream, a second non-nested SEI message, the second non-nested SEI message being a buffering period SEI message that indicates initial delays for HRD operations, a picture timing SEI message that indicates picture output times and picture/sub-picture removal times for HRD operations, or a decoding unit SEI message that indicates sub-picture removal times for HRD operations;
determining an operation point applicable to the second non-nested SEI message,
wherein a highest temporal identifier specified in all NAL units of the operation point applicable to the second non-nested SEI message is equal to a temporal identifier specified in a SEI NAL unit that encapsulates the second non-nested SEI message, and
wherein a set of layer identifiers of all the NAL units of the operation point applicable to the second non-nested SEI message contains all integer values in the range of 0 to the layer identifier specified in the SEI NAL unit that encapsulates the second non-nested SEI message; and

performing, based in part on values of one or more syntax elements of the second non-nested SEI message, an HRD operation with regard to the operation point applicable to the second non-nested SEI message.

8. A video processing device comprising one or more processors configured to:
obtain, from a bitstream of coded video data that includes a sequence of network abstraction layer (NAL)

Assignees

• Qualcomm Inc

Inventors

• Wang Ye-Kui
• Chen Ying
• Ramasubramonian Adarsh Krishnan

Classifications

• H04N19/70 Primary

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/30

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N13/0048 Primary

  Electricity · mapped topic

• H04N13/161 Primary

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title