High-level signalling for fisheye video data

Patent metadata

Field	Value
Publication number	US-2018343472-A1
Application number	US-201815987231-A
Country	US
Kind code	A1
Filing date	May 23, 2018
Priority date	May 25, 2017
Publication date	Nov 29, 2018
Grant date	—

Abstract

An example method includes processing a file including fisheye video data, the file including a syntax structure including a plurality of syntax elements that specify attributes of the fisheye video data, wherein the plurality of syntax elements includes: a first syntax element that explicitly indicates whether the fisheye video data is monoscopic or stereoscopic, and one or more syntax elements that implicitly indicate whether the fisheye video data is monoscopic or stereoscopic; determining, based on the first syntax element, whether the fisheye video data is monoscopic or stereoscopic; and rendering, based on the determination, the fisheye video data as monoscopic or stereoscopic.

First claim

What is claimed is:

1 . A method of processing a file including video data, the method comprising:
processing a file including fisheye video data, the file including a syntax structure including a plurality of syntax elements that specify attributes of the fisheye video data, wherein the plurality of syntax elements includes: a first syntax element that explicitly indicates whether the fisheye video data is monoscopic or stereoscopic, and one or more syntax elements that implicitly indicate whether the fisheye video data is monoscopic or stereoscopic; determining, based on the first syntax element, whether the fisheye video data is monoscopic or stereoscopic; and outputting, based on the determination, the fisheye video data for rendering as monoscopic or stereoscopic.

2 . The method of claim 1 , wherein the first syntax element is included in a set of initial bits of the syntax structure.

3 . The method of claim 2 , wherein the set of initial bits is 24 bits long.

4 . The method of claim 1 , wherein the file includes a box that includes the syntax structure.

5 . The method of claim 4 , wherein the box is a first box that is included in a second box that includes scheme information, the method further comprising:
determining whether the first box includes a third box that indicates whether pictures of the fisheye video data are packed region-wise.

6 . The method of claim 5 , further comprising:
in response to determining that the first box includes the third box, unpacking the pictures of the fisheye video data prior to rendering the pictures of the fisheye video data; or in response to determining that the first box does not include the third box, rendering the pictures of the fisheye video data without unpacking the pictures of the fisheye video data.

7 . The method of claim 5 , wherein the first box is a SchemeInformationBox, the second box is a FisheyeOmnidirectionalVideoBox, and the third box is a RegionWisePackingBox.

8 . The method of claim 1 , wherein the syntax structure further includes a second syntax element that specifies a number of circular images included in each picture of the fisheye video data.

9 . The method of claim 8 , wherein the syntax structure comprises, for each respective circular image, a respective third syntax element that indicates a view identifier of the respective circular image.

10 . The method of claim 1 , wherein the syntax structure is external to video coding layer (VCL) data encapsulated by the file.

11 . The method of claim 1 , wherein determining whether the fisheye video data is monoscopic or stereoscopic comprises:
determining, based on the first syntax element and regardless of the syntax elements that implicitly indicate whether the fisheye video data is monoscopic or stereoscopic, whether the fisheye video data is monoscopic or stereoscopic.

12 . A method for generating a file including video data, the method comprising:
obtaining fisheye video data and extrinsic parameters of cameras used to capture the fisheye video data; determining, based on the extrinsic parameters, whether the fisheye video data is monoscopic or stereoscopic; and encoding, in a file, the fisheye video data and a syntax structure including a plurality of syntax elements that specify attributes of the fisheye video data, wherein the plurality of syntax elements includes: a first syntax element that explicitly indicates whether the fisheye video data is monoscopic or stereoscopic, and one or more syntax elements that explicitly indicate the extrinsic parameters of the cameras used to capture the fisheye video data.

13 . The method of claim 12 , wherein encoding the first syntax element comprises encoding the first syntax element in a set of initial bits of the syntax structure.

14 . The method of claim 13 , wherein the set of initial bits is 24 bits long.

15 . The method of claim 12 , wherein the file includes a box that includes the syntax structure.

16 . The method of claim 15 , wherein the box is a first box that is included in a second box that includes scheme information, the method further comprising:
encoding, in the first box, a third box that indicates whether pictures of the fisheye video data are packed region-wise.

17 . The method of claim 16 , wherein the first box is a SchemeInformationBox, the second box is a FisheyeOmnidirectionalVideoBox, and the third box is a RegionWisePackingBox.

18 . The method of claim 12 , wherein the syntax structure further includes a second syntax element that specifies a number of circular images included in each picture of the fisheye video data.

19 . The method of claim 18 , wherein the syntax structure comprises, for each respective circular image, a respective third syntax element that indicates a view identifier of the respective circular image.

20 . The method of claim 12 , wherein the fisheye video data is encoded in a video coding layer (VCL) and wherein the syntax structure is external to the VCL.

21 . A device for processing video data, the device comprising:
a memory configured to store at least a portion of a file including fisheye video data, the file including a syntax structure including a plurality of syntax elements that specify attributes of the fisheye video data, wherein the plurality of syntax elements includes: a first syntax element that explicitly indicates whether the fisheye video data is monoscopic or stereoscopic, and one or more syntax elements that implicitly indicate whether the fisheye video data is monoscopic or stereoscopic; and one or more processors configured to:
determine, based on the first syntax element, whether the fisheye video data is monoscopic or stereoscopic; and
output, based on the determination, the fisheye video data for rendering as monoscopic or stereoscopic.

22 . The device of claim 21 , wherein the first syntax element is included in a set of initial bits of the syntax structure.

23 . The device of claim 22 , wherein the set of initial bits is 24 bits long.

24 . The device of claim 21 , wherein the file includes a box that includes the syntax structure.

25 . The device of claim 24 , wherein the box is a first box that is included in a second box that includes scheme information, the one or more processors are further configured to:
determine whether the first box includes a third box that indicates whether pictures of the fisheye video data are packed region-wise.

26 . The device of claim 25 , wherein:
in response to determining that the first box includes the third box, the one or more processors are further configured to, unpack the pictures of the fisheye video data prior to rendering the pictures of the fisheye video data; or in response to determining that the first box does not include the third box, the one or more processors are further configured to, render the pictures of the fisheye video data without unpacking the pictures of the fisheye video data.

27 . The device of claim 25 , wherein the first box is a SchemeInformationBox, the second box is a FisheyeOmnidirectionalVideoBox, and the third box is a RegionWisePackingBox.

28 . The device of claim 21 , wherein the syntax structure further includes a second syntax element that specifies a number of circular images included in each picture of the fisheye video data.

29 . The device of

Assignees

• Qualcomm Inc

Inventors

• Wang Ye-Kui
• Bi Ning
• Forutanpour Bijan

Classifications

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title

• H04N21/85406

  involving a specific file format, e.g. MP4 format · CPC title

• H04N19/70 Primary

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title