Bitstream signaling of error mitigation in sub-picture bitstream based viewport dependent video coding

Patent metadata

Field	Value
Publication number	US-2021021814-A1
Application number	US-202017061416-A
Country	US
Kind code	A1
Filing date	Oct 1, 2020
Priority date	Apr 3, 2018
Publication date	Jan 21, 2021
Grant date	—

Abstract

A video coding mechanism for viewpoint dependent video coding is disclosed. The mechanism includes mapping a spherical video sequence into a plurality of sub-picture video sequences. The mechanism further includes encoding the plurality of sub-picture video sequences as sub-picture bitstreams to support merging of the plurality of sub-picture bitstreams, the encoding ensuring that each sub-picture bitstream is self-referenced and two or more of the sub-picture bitstreams can be merged to generate a single video bitstream using a lightweight bitstream rewriting process that does not involve changing of any block-level coding results. A mergable indication is encoded to indicate that the sub-picture bitstream containing the indication is compatible with a multi-bitstream merge function for reconstruction of the spherical video sequence. A set of the sub-picture bitstreams and the mergable indication are transmitted toward the decoder to support decoding and displaying a virtual reality video viewport.

First claim

What is claimed is:

1 . A method implemented in a video coding device, the method comprising:
mapping, by a processor of the video coding device, a spherical video sequence, into a plurality of sub-picture video sequences each containing a sequence of sub-pictures; encoding, by the processor, the plurality of sub-picture video sequences as sub-picture bitstreams to support merging of a plurality of sub-picture bitstreams according to a constraint that each sub-picture bitstream is self-referenced and two or more of the sub-picture bitstreams can be merged to generate a single video bitstream using a lightweight bitstream rewriting process that does not involve changing of any block-level coding results; encoding, by the processor, a mergable indication within one of the sub-picture bitstreams, the mergable indication indicating that the sub-picture bitstream containing the mergable indication is compatible with a multi-bitstream merge function for reconstruction of the spherical video sequence or a part thereof; and transmitting, by a transmitter of the video coding device, a set of the sub-picture bitstreams and the mergable indication toward a decoder to support decoding and displaying a virtual reality video viewport.

2 . The method of claim 1 , wherein the mergable indication is included in a supplemental enhancement information (SEI) message.

3 . The method of claim 1 , wherein the mergable indication is included in a temporal motion constrained tile set (MCTS) SEI message, wherein the temporal MCTS SEI message is used despite inclusion of a single tile in a picture of the spherical video sequence, and wherein the single tile is an MCTS.

4 . The method of claim 3 , wherein the temporal MCTS SEI message has an each_tile_one_tile_set_flag set equal to 1.

5 . The method of claim 2 , wherein the SEI message is a lightweight merging indication SEI message.

6 . The method of claim 2 , wherein the SEI message further indicates that waveform parallel processing is disabled by setting an entropy_coding_sync_enabled_flag to zero in picture parameter sets (PPSs) for each sub-picture bitstream.

7 . The method of claim 1 , wherein the mergable indication is a syntax element contained as video usability information (VUI) in a sequence parameter set (SPS).

8 . The method of claim 1 , wherein the mergable indication is a syntax element contained as video usability information (VUI) in a video parameter set (VPS).

9 . The method of claim 1 , wherein the mergable indication is included between a set of general profile compatibility flags and a general level identification code in a profile tier level syntax structure.

10 . A video coding device comprising:
a processor configured to:
map a spherical video sequence into a plurality of sub-picture video sequences each containing sub-pictures;
encode the plurality of sub-picture video sequences as sub-picture bitstreams to support merging of a plurality of sub-picture bitstreams according to a constraint that each sub-picture bitstream is self-referenced and two or more of the sub-picture bitstreams can be merged to generate a single video bitstream using a lightweight bitstream rewriting process that does not involve changing of any block-level coding results; and
encode a mergable indication within at least one of the sub-picture bitstreams, the mergable indication indicating that the sub-picture bitstream containing the indication is compatible with a multi-bitstream merge function for reconstruction of the spherical video sequence or a part thereof; and
a transmitter coupled to the processer, the transmitter configured to transmit a set of the sub-picture bitstreams and the mergable indication toward a decoder to support decoding and displaying a virtual reality video viewport.

11 . The video coding device of claim 10 , wherein the mergable indication is included in a supplemental enhancement information (SEI) message.

12 . The video coding device of claim 10 , wherein the mergable indication is included in a temporal motion constrained tile set (MCTS) SEI message, wherein the temporal MCTS SEI message is used despite inclusion of a single tile in a picture of the spherical video sequence, and wherein the single tile is an MCTS.

13 . The video coding device of claim 12 , wherein the temporal MCTS SEI message has an each_tile_one_tile_set_flag set equal to 1.

14 . The video coding device of claim 11 , wherein the SEI message is a lightweight merging indication SEI message.

15 . The video coding device of claim 11 , wherein the SEI message further indicates that waveform parallel processing is disabled by setting an entropy_coding_sync_enabled_flag to zero in picture parameter sets (PPS) for each sub-picture bitstream.

16 . The video coding device of claim 10 , wherein the mergable indication is a syntax element contained as video usability information (VUI) in a sequence parameter set (SPS).

17 . The video coding device of claim 10 , wherein the mergable indication is a syntax element contained as video usability information (VUI) in a video parameter set (VPS).

18 . The video coding device of claim 10 , wherein the mergable indication is included between a set of general profile compatibility flags and a general level identification code flag in a profile tier level syntax structure.

19 . A method comprising:
employing, by a processor of an encoder, a temporal motion-constrained tile set (MCTS) supplemental enhancement information (SEI) message for a picture; setting, by the processor, a tiles_enabled_flag to zero for an active picture parameter set (PPS) to indicate the picture contains only one tile; and setting, by the processor, a value of each_tile_one_tile_set_flag to one.

20 . The method of claim 19 , wherein the temporal MCTS SEI message is used despite inclusion of a single tile in a picture of a spherical video sequence.

21 . The method of claim 19 , further comprising storing, by a memory, the temporal MCTS SEI message, the PPS, and the each_tile_one_tile_set_flag as part of an encoded video bitstream.

22 . The method of claim 19 , further comprising encoding the picture into sub-picture bitstreams carried in tracks.

23 . The method of claim 22 , wherein encoding further comprises selecting motion vectors that reference fractional sample locations that employ only full sample locations inside a same sub-picture video signal for interpolation, and not selecting motion vectors that reference fractional sample locations that employ partial sample locations inside the same sub-picture video signal for interpolation.

24 . The method of claim 22 , wherein encoding further comprises selecting motion vectors that reference full sample locations in a same sub-picture video signal for interpolation, and not selecting motion vectors that reference full sample locations outside the same sub-picture video signal for interpolation.

Assignees

• Huawei Tech Co Ltd

Inventors

• Wang Ye-Kui
• Fang Huameng
• Fan Yuqun
• Chen Jianle

Classifications

• H04N19/119

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• G06T9/00

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title