Client-based adaptive streaming of nonlinear media

Patent metadata

Field	Value
Publication number	US-11025919-B2
Application number	US-201816139694-A
Country	US
Kind code	B2
Filing date	Sep 24, 2018
Priority date	Oct 3, 2017
Publication date	Jun 1, 2021
Grant date	Jun 1, 2021

Abstract

A method for adaptive streaming of nonlinear video is described wherein the method comprises receiving a manifest file comprising metadata defining a playback period defining video data of a predetermined playback duration, a playback period including at least one adaptation set defining a plurality of representations, each representation defining a set of segment identifiers, each segment identifier identifying a segment comprising a sequence of video frames; the metadata further comprising spatial playback information including at least a first region descriptor associated with a first adaptation set of a first playback period, the first region descriptor defining a destination identifier and a first region in video frames of the first adaptation set, the destination identifier signaling the client apparatus that the first region is linked to a second adaptation set; the client apparatus using the metadata to playback video data of the first adaptation set by requesting one or more network nodes to transmit segments of the first adaptation set; and, receiving or determining, during playback of requested segments, information associated with at least a second region in the video frames of the first adaptation set, the second region defining a region of interest (ROI) of a user of the client apparatus; the client apparatus selecting the second adaptation set for playback after the end of the playback of the first adaptation set on the basis of the first and second region, the selecting being based on a distance or an overlap between the first and second region.

First claim

The invention claimed is:

1. A method for adaptive streaming of nonlinear video comprising:
a client apparatus receiving a first manifest file comprising metadata defining one or more playback periods, a playback period defining video data of a predetermined playback duration, a playback period including at least one adaptation set defining a plurality of representations, each representation defining a set of segment identifiers, each segment identifier identifying a segment comprising video data, the video data including a sequence of video frames;
the metadata further comprising spatial playback information including at least a first region descriptor associated with a first adaptation set of a first playback period, the first region descriptor defining a destination identifier and a first region, the first region encompassed by video frames, the video frames belonging to the first adaptation set, the destination identifier signaling the client apparatus that the first region is linked to a second adaptation set, the second adaptation set being associated with a second region descriptor, the second region descriptor including a source identifier that matches the destination identifier;
the client apparatus using the metadata to playback video data of the first adaptation set by requesting one or more network nodes to transmit segments of the first adaptation set, and by receiving, during playback of requested segments, information associated with at least a second region, the second region encompassed by the video frames of the first adaptation set, the second region defining a region of interest (ROI) of a user of the client apparatus; and
the client apparatus selecting the second adaptation set for playback after the end of the playback of the first adaptation set on the basis of the first and second region, the selecting being based on a distance or an overlap between the first and second region.

2. The method according to claim 1 , wherein the first region descriptor further comprises a period identifier, the period identifier signaling the client apparatus that the second adaptation set is defined in the metadata of a second playback period.

3. The method according to claim 1 , wherein the first region descriptor further includes location information of the first region and information associated with dimensions and/or shape of the first region, the location information being coordinates of the first region in the video frames of the first adaptation set, or information to determine coordinates of the first region in the video frames of the first adaptation set.

4. The method according to claim 1 , wherein the spatial playback information further includes a dynamic region parameter, the dynamic region parameter signaling the client apparatus that the location of the first region changes as a function of time, the coordinates associated with the location of the first region being stored as metadata of the video data of the first playback period.

5. The method according to claim 1 , wherein the first adaptation set is associated with a plurality of region descriptors defining locations of a plurality of regions in the video frames of the first adaptation set, each of the region descriptors signaling the client apparatus that each of the plurality of regions is linked to a further adaptation set, and wherein the client apparatus selecting the second adaptation set comprises the selecting being based on the locations of the plurality of regions and the location of the ROI of the user of the client apparatus.

6. The method according claim 1 , wherein the first manifest file further comprises metadata defining the second adaptation set and, the method further comprises:
the client apparatus requesting one or more network nodes to transmit segments of the second adaptation set.

7. The method according to claim 1 , wherein the spatial playback information further includes a manifest file reference identifier, the manifest file reference identifier including information defining a location of a network node, the manifest file reference identifier signaling the client apparatus to request the network node to transmit metadata of the second adaptation set to the client device, the manifest file reference identifier defining a MPD chaining element associated with the second adaptation set.

8. The method according to claim 7 , further comprising:
the client apparatus using the manifest file reference identifier to request a network node to transmit a second manifest file comprising metadata of the second adaptation set, the metadata including segment identifiers associated with the second adaptation set; and
the client apparatus using the second manifest file to request a network node to transmit segments of the second adaptation set.

9. Method according claim 1 , wherein the adaptive streaming is based on an HTTP adaptive streaming protocol, the protocol being one of: MPEG Dynamic Adaptive Streaming over HTTP (MPEG DASH), 3GPP DASH or MPEG Common Media Application Format (CMAF); and wherein a playback period is defined as a Period as defined in a MPEG DASH standard or as a CMAF Presentation as defined in the CMAF standard, a Period comprising one or more Adaptation Sets, wherein an Adaptation Set includes one or more Representations, a Representation including segments comprising video data of a predetermined video quality and CMAF Presentation comprising one or more CMAF Switching Sets, a CMAF Switching Set including one or more CMAF Tracks, a Track including CMAF fragments comprising video data of a predetermined video quality.

10. The method according to claim 7 , further comprising:
the client apparatus using the manifest file reference identifier to request a network node to transmit a second manifest file comprising metadata of the second adaptation set, the metadata including information for determining segment identifiers associated with the second adaptation set; and
the client apparatus using the second manifest file to request a network node to transmit segments of the second adaptation set.

11. The method according to claim 1 , wherein the spatial playback information further includes information for determining a manifest file reference identifier, the manifest file reference identifier including information defining a location of a network node, the manifest file reference identifier signaling the client apparatus to request the network node to transmit metadata of the second adaptation set to the client device, the manifest file reference identifier defining a MPD chaining element associated with the second adaptation set.

12. The method according to claim 11 , further comprising:
the client apparatus using the manifest file reference identifier to request a network node to transmit a second manifest file comprising metadata of the second adaptation set, the metadata including segment identifiers associated with the second adaptation set; and
the client apparatus using the second manifest file to request a network node to transmit segments of the second adaptation set.

13. The method according to claim 11 , further comprising:
the client apparatus using the manifest file reference identifier to request a network node to transmit a second manifest file comprising metadata of the second adaptation set, the metadata including information for determining segment identifiers associated with the second adaptation set; and
the client apparatus using the second manifest file to request a network node to transmit segments of the second adaptation set.

14. A client apparatus comprising:
a computer readable storage medium having compute

Assignees

• Koninklijke Kpn Nv
• Nederlandse Organisatie Voor Toegepast—Natuurwetenschappelijk Onderzoek Tno

Inventors

• Da Silva Pratas Gabriel Alexandre
• Thomas Emmanuel Didier Remi

Classifications

• H04N21/4728 Primary

  for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region · CPC title

• H04L65/612

  for unicast · CPC title

• H04L65/762

  at the source  (reformatting of additional data in video distribution servers H04N21/2355) · CPC title

• H04N21/6587

  Control parameters, e.g. trick play commands, viewpoint selection · CPC title

• G06F16/78

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title