Method and apparatus for streaming multimedia data with access point positioning information

Patent metadata

Field	Value
Publication number	US-9813740-B2
Application number	US-201213593680-A
Country	US
Kind code	B2
Filing date	Aug 24, 2012
Priority date	Aug 24, 2012
Publication date	Nov 7, 2017
Grant date	Nov 7, 2017

Abstract

A method and apparatus for streaming multimedia data over a network is provided. A playlist file for a selected multimedia presentation is downloaded by a client device over the network from a streaming server using a transfer protocol. The playlist file provides a list of URIs for media files each providing compressed video data corresponding to a segment of the multimedia presentation. The compressed video data includes reference frames used for video compression. A data file that is associated with the playlist file is also downloaded by the client device over the network from the streaming server using a transfer protocol and provides information identifying a location, a size, and a time stamp for each the reference frames contained in the media files listed by the playlist file. This information can be used by the client device to perform trick plays and to switch and stitch between variant streams.

First claim

We claim:

1. A method comprising:
downloading, by a client device, a playlist file for a multimedia presentation over a network from a streaming server using a transfer protocol, the playlist file including an ordered list of Universal Resource Indicators (URIs) for a plurality of media files, each of the plurality of media files including compressed video data corresponding to a segment of the multimedia presentation and containing reference frames;
downloading, by the client device, a data file that is associated with the playlist file, the data file including, for each of a plurality of reference frames included in a media file from the plurality of media files, reference frame information indicating a location of the reference frame in the media file;
displaying, by the client device, the multimedia presentation based on video data received from the streaming server and included in the media file;
responsive to a user requesting to perform a trick play related to the displaying of the multimedia presentation, transmitting, by the client device to the streaming server, a request for the plurality of reference frames, the request including the reference frame information for each of the plurality of reference frames included in the data file;
responsive to transmitting the request, receiving, by the client device from the streaming server, the plurality of reference frames identified by the streaming server in the media file based on the reference frame information;
performing the trick play using one or more reference frames from the plurality of reference frames, wherein downloading the playlist file includes downloading a plurality of playlist files providing a plurality of variant streams for the multimedia presentation with each of the plurality of variant streams providing an encoding of the multimedia presentation at a different bit rate, and a separate data file is downloaded for each of the of the plurality of variant streams;
determining available bandwidth of the network;
selecting, based on the available bandwidth from the plurality of variant streams, a first variant stream for use in playing the multimedia presentation;
downloading media files associated with the first variant stream;
reassembling segments included in the downloaded media files to play the multimedia presentation as a continuous stream;
selecting, from the plurality of variant streams, a second variant stream for playing the multimedia presentation; and
switching between the first variant stream and the second variant stream by aligning corresponding reference frames of the first and second variant streams based on time stamps of the reference frames included in the data file for each of the first and second variant streams.

2. The method of claim 1 , wherein the transfer protocol is Hypertext Transfer Protocol (HTTP) Live Streaming (HLS), and the downloading of the playlist file and the data file is performed by the client device electronically as HTTP fetches.

3. The method of claim 1 , wherein the client device is selected from a group consisting of a set-top box, a server, a gateway, a computer, an IP client device, a wireless electronic device, a tablet, a laptop computer, and a smartphone.

4. The method of claim 1 , wherein each of the plurality of reference frames is an Instantaneous Decoder Refresh (IDR) frame within the compressed video data.

5. The method of claim 1 , wherein the reference frame information includes a size and time stamp of the reference frame.

6. The method of claim 1 , wherein the switching occurs at a reference frame located at an intermediate position within one of the segments during playing thereof.

7. The method of claim 1 , wherein the trick play is to fast forward through the multimedia presentation.

8. The method of claim 1 , wherein the trick play is to fast reverse through the multimedia presentation.

9. The method of claim 1 , wherein the trick play is to seek in the multimedia presentation.

10. A client device for processing multimedia data streamed over a network, comprising at least one processing unit and a non-transitory computer readable storage medium containing computer instructions that when executed by the at least one processing unit cause the processing unit to:
download a playlist file for a multimedia presentation over the network from a streaming server using a transfer protocol, the playlist file including an ordered list of Universal Resource Indicators (URIs) for a plurality of media files, each of the plurality of media files including compressed video data corresponding to a segment of the multimedia presentation and containing reference frames;
download a data file that is associated with the playlist file, the data file including, for each of a plurality of reference frames included in a media file from the plurality of media files, reference frame information indicating a location of the reference frame in the media file;
display the multimedia presentation based on video data received from the streaming server and included in the media file;
responsive to a user requesting to perform a trick play related to the display of the multimedia presentation, transmit, to the streaming server, a request for the plurality of reference frames, the request including the reference frame information for each of the plurality of reference frames included in the data file;
responsive to transmitting the request, receive, from the streaming server, the plurality of reference frames identified by the streaming server in the media file based on the reference frame information;
perform the trick play using one or more reference frames from the plurality of reference frames, wherein downloading the playlist file includes downloading a plurality of playlist files providing a plurality of variant streams for the multimedia presentation with each of the plurality of variant streams providing an encoding of the multimedia presentation at a different bit rate, and a separate data file is downloaded for each of the of the plurality of variant streams;
determine available bandwidth of the network;
select, based on the available bandwidth from the plurality of variant streams, a first variant stream for use in playing the multimedia presentation;
download media files associated with the first variant stream;
reassemble segments included in the downloaded media files to play the multimedia presentation as a continuous stream;
select, from the plurality of variant streams, a second variant stream for playing the multimedia presentation; and
switch between the first variant stream and the second variant stream by aligning corresponding reference frames of the first and second variant streams based on time stamps of the reference frames included in the data file for each of the first and second variant streams.

11. The client device of claim 10 , wherein the transfer protocol is Hypertext Transfer Protocol (HTTP) Live Streaming (HLS), wherein the client device is selected from a group consisting of a set-top box, a server, a gateway, a computer, an IP client device, a wireless electronic device, a tablet, a laptop computer, and a smartphone, and wherein each of the plurality of reference frames is an Instantaneous Decoder Refresh (IDR) frame within the compressed video data.

12. A method of streaming multimedia data over a network, the comprising:
transmitting, by a computer system, a playlist file for a multimedia presentation to a client device over the network using a transfer protocol, the playlist file identifying an ordered list of Universal Resource Indicators (URIs) for a plurality of media files, each of the plurality of media files inclu

Assignees

• Panje Krishna Prasad
• Del Sordo Christopher S
• Tumuluru Sundar Murthy
• Google Inc

Inventors

• Panje Krishna Prasad
• Del Sordo Christopher S
• Tumuluru Sundar Murthy

Classifications

• H04N21/44209

  Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896) · CPC title

• H04N21/26258

  for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40) · CPC title

• H04N21/234363 Primary

  by altering the spatial resolution, e.g. for clients with a lower screen resolution · CPC title

• H04N21/8456

  by decomposing the content in the time domain, e.g. in time segments · CPC title

• H04N21/23439

  for generating different versions · CPC title