Error mitigation in sub-picture bitstream based viewport dependent video coding

Patent metadata

Field	Value
Publication number	US-11405606-B2
Application number	US-202017061421-A
Country	US
Kind code	B2
Filing date	Oct 1, 2020
Priority date	Apr 3, 2018
Publication date	Aug 2, 2022
Grant date	Aug 2, 2022

Abstract

A video coding mechanism for viewpoint dependent video coding is disclosed. The mechanism includes receiving a spherical video signal stitched from multiple directional video streams. The spherical video signal is mapped into a plurality of sub-picture video signals each containing a sequence of sub-pictures. The plurality of sub-picture video signals, are encoded as a plurality of sub-picture bitstreams, such that, when decoded at a decoder, a value of each sample in each sub-picture is identical to a value of a corresponding sample in a decoded entire picture composed from the sub-pictures. The plurality of sub-picture bitstreams are composed into a plurality of sub-picture tracks with an indication that the sub-picture tracks are conforming to a particular video profile. The sub-picture bitstreams are transmitted toward a decoder to support decoding and displaying virtual reality video viewport.

First claim

What is claimed is:

1. A method implemented by an encoder, the method comprising:
encoding, by a processor at the encoder, pictures in sub-picture bitstreams in tracks by:
ensuring no sample values outside a sub-picture are referenced for inter-prediction;
restricting motion vectors to point to full-sample locations inside the sub-picture and to fractional-sample locations that employ only the full-sample locations inside the sub-picture for interpolation;
restricting inter-prediction so motion vector candidates for temporal motion vector prediction are not derived from blocks outside the sub-picture; and
requiring that samples with a same sample number across a plurality of tracks have a same presentation time.

2. The method of claim 1 , wherein encoding pictures in sub-picture bitstreams in tracks further includes that when multiple tiles are present in a first sub-picture bitstream, a value of an entropy_coding_sync_enabled_flag in a picture parameter set (PPS) is set to zero for each sub-picture bitstream carried in a corresponding track.

3. The method of claim 1 , wherein encoding pictures in sub-picture bitstreams in tracks further includes requiring tracks to contain a same number of media samples.

4. The method of claim 1 , wherein encoding pictures in sub-picture bitstreams in tracks further includes that sub-pictures carried in media samples with the same sample number across the tracks have a same picture order count value.

5. A video coding device comprising:
a processor configured to encode pictures in sub-picture bitstreams in tracks by:
ensuring no sample values outside a sub-picture are referenced for inter-prediction;
restricting motion vectors to point to full-sample locations inside the sub-picture and to fractional-sample locations that employ only the full-sample locations inside the sub-picture for interpolation;
restricting inter-prediction so motion vector candidates for temporal motion vector prediction are not derived from blocks outside the sub-picture; and
requiring that samples with a same sample number across a plurality of tracks have a same presentation time.

6. The video coding device of claim 5 , wherein encoding pictures in sub-picture bitstreams in tracks further includes that when multiple tiles are present in a first sub-picture bitstream, a value of an entropy_coding_sync_enabled_flag in a picture parameter set (PPS) is set to zero for each sub-picture bitstream carried in a corresponding track.

7. The video coding device of claim 5 , wherein encoding pictures in sub-picture bitstreams in tracks further includes requiring tracks to contain a same number of media samples.

8. The video coding device of claim 5 , wherein encoding pictures in sub-picture bitstreams in tracks further includes that sub-pictures carried in media samples with the same sample number across the tracks have a same picture order count value.

9. A non-transitory computer readable medium comprising a computer program product for use by a video coding device, the computer program product comprising computer executable instructions stored on the non-transitory computer readable medium such that when executed by a processor cause the video coding device to:
encode pictures in sub-picture bitstreams in tracks by:
ensuring no sample values outside a sub-picture are referenced for inter-prediction;
restricting motion vectors to point to full-sample locations inside the sub-picture and to fractional-sample locations that employ only the full-sample locations inside the sub-picture for interpolation;
restricting inter-prediction so motion vector candidates for temporal motion vector prediction are not derived from blocks outside the sub-picture; and
requiring that sub-pictures carried in media samples with a same sample number across the tracks have a same picture order count value.

10. The non-transitory computer readable medium of claim 9 , wherein encoding pictures in sub-picture bitstreams in tracks further includes that when multiple tiles are present in a first sub-picture bitstream, a value of an entropy_coding_sync_enabled_flag in a picture parameter set (PPS) is set to zero for each sub-picture bitstream carried in a corresponding track.

11. The non-transitory computer readable medium of claim 9 , wherein encoding pictures in sub-picture bitstreams in tracks further includes requiring tracks to contain a same number of media samples.

12. The non-transitory computer readable medium of claim 9 , wherein encoding pictures in sub-picture bitstreams in tracks further includes requiring that samples with the same sample number across a plurality of tracks have a same presentation time.

Assignees

• Huawei Tech Co Ltd

Inventors

• Wang Ye-Kui
• Fang Huameng
• Fan Yuqun
• Chen Jianle

Classifications

• H04N19/119

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title

• H04N19/105 Primary

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/44

  Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title