Dash-based streaming of point cloud content based on recommended viewports

Patent metadata

Field	Value
Publication number	US-11729243-B2
Application number	US-202017027524-A
Country	US
Kind code	B2
Filing date	Sep 21, 2020
Priority date	Sep 20, 2019
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Various embodiments herein provide adaptive streaming mechanisms for distribution of point cloud content. The point cloud content may include immersive media content in a dynamic adaptive streaming over hypertext transfer protocol (DASH) format. Various embodiments provide DASH-based mechanisms to support viewport indication during streaming of volumetric point cloud content. Other embodiments may be described and claimed.

First claim

The invention claimed is:

1. One or more non-transitory, computer-readable media (NTCRM) having instructions, stored thereon, that when executed by one or more processors cause a user equipment (UE) to:
receive a manifest file from a server, the manifest file including hierarchical levels, the hierarchical levels including information characterizing encoded portions of immersive media content available for adaptive streaming from different viewports, and the hierarchical levels including an adaptation set level encompassing one or more other levels;
determine recommended viewport information from the adaptation set level indicated by the manifest file, wherein the recommended viewport information in the manifest file includes syntax elements, a point cloud object identifier, and point cloud context information;
identify a selected viewport based on the recommended viewport information; and
dynamically switch streaming between different encoded portions of the immersive media content based on the selected viewport.

2. The one or more NTCRM of claim 1 , wherein the immersive media content is in a dynamic adaptive streaming over hypertext transfer protocol (DASH) content format, and wherein the manifest file is a DASH media presentation description (MPD).

3. The one or more NTCRM of claim 2 , wherein the instructions, when executed, further cause the UE to:
determine a user selected viewport;
select one or more DASH representations based on a recommended viewport indicated by the recommended viewport information that corresponds to the user selected viewport; and
encode, for transmission to the server, a request for the selected one or more DASH representations.

4. The one or more NTCRM of claim 1 , wherein the immersive media content is point cloud video content.

5. The one or more NTCRM of claim 1 , wherein the recommended viewport information in the manifest file includes a plurality of recommended viewports associated with respective different versions of the immersive media content.

6. The one or more NTCRM of claim 1 , wherein the recommended viewport information indicates a three-dimensional (3D) viewport corresponding to a spherical region of the immersive media content, or a two-dimensional (2D) viewport corresponding to a planar region of the immersive media content.

7. The one or more NTCRM of claim 1 , wherein the selected viewport is identified using one or more sensors and/or control mechanisms of the UE.

8. The one or more NTCRM of claim 1 , wherein the syntax elements include a center azimuth, a center elevation, a center tilt, an azimuth range, and an elevation range of the viewport, and wherein the syntax elements further include coordinates of a center point of a sphere that contains the viewport.

9. The one or more NTCRM of claim 1 , wherein the recommended viewport information includes contextual information that includes one or more of: a string value associated with each adaptation set level to carry relevant contextual information; an object identifier (ID) value indicating a specific point cloud object from which the recommended viewport is to be derived; and/or the a patch ID value indicating a specific point cloud patch from which the recommended viewport is to be derived.

10. One or more non-transitory, computer-readable media (NTCRM) having instructions, stored thereon, that when executed by one or more processors cause a server to:
store a manifest file that defines hierarchical levels including an adaptation set level encompassing one or more other levels, the hierarchical levels including information characterizing encoded portions of immersive media content available for adaptive streaming from different viewports;
send, to a client operating at a user equipment (UE), recommended viewport information from the adaptation set level of the manifest file, wherein the recommended viewport information in the manifest file includes syntax elements, a point cloud object identifier, and point cloud context information;
receive, from the client, an indication of a user-selected viewport; and
stream different encoded portions of the immersive media content to the client based on the user-selected viewport.

11. The one or more NTCRM of claim 10 , wherein the immersive media content is in a dynamic adaptive streaming over HTTP (DASH) content format, and wherein the manifest file is a DASH media presentation description (MPD).

12. The one or more NTCRM of claim 11 , wherein the indication of the user-selected viewport includes one or more DASH representations based on the recommended viewport information indicated by the MPD that match the user-selected viewport; and the different encoded portions of the immersive media content are the one or more DASH representations.

13. The one or more NTCRM of claim 10 , wherein the immersive media content is point cloud video content.

14. The one or more NTCRM of claim 10 , wherein different versions of the immersive media content are stored at the server or associated storage systems, and each of the different versions are produced and encoded for different recommended viewports indicated by the manifest file.

15. The one or more NTCRM of claim 10 , wherein the recommended viewport information indicates a three-dimensional (3D) viewport corresponding to a spherical region of the immersive media content.

16. The one or more NTCRM of claim 10 , wherein the recommended viewport information indicates a two-dimensional (2D) viewport corresponding to a planar region of the immersive media content.

17. The one or more NTCRM of claim 10 , wherein the syntax elements include a center azimuth, a center elevation, a center tilt, an azimuth range, and an elevation range of the viewport, and wherein the syntax elements further include coordinates of a center point of a sphere that contains the viewport.

18. The one or more NTCRM of claim 10 , wherein the recommended viewport information includes contextual information that includes one or more of: a string value associated with each adaptation set level to carry relevant contextual information; an object identifier (ID) value indicating a specific point cloud object from which a recommended viewport is to be derived; and/or the a patch ID value indicating a specific point cloud patch from which the recommended viewport is to be derived.

Assignees

• Intel Corp

Inventors

• Oyman Ozgur
• Yeung Fai
• Gill Harleen
• Loza Kimberly

Classifications

• H04L65/612

  for unicast · CPC title

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title

• H04L67/02

  based on web technology, e.g. hypertext transfer protocol [HTTP] · CPC title

• H04L65/756

  adapting media to device capabilities · CPC title

• H04L65/80 Primary

  Responding to QoS · CPC title