Using time-stamped event entries to facilitate synchronizing data streams

Patent metadata

Field	Value
Publication number	US-9794605-B2
Application number	US-77050707-A
Country	US
Kind code	B2
Filing date	Jun 28, 2007
Priority date	Jun 28, 2007
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

An embodiment of a system to synchronize data streams is described. During operation, the system receives a first data stream having a first data type and performs a sequence of operations on at least a portion of the first data stream. Next, the system stores metadata associated with the first data stream one or more times during the sequence of operations. Metadata can include a time stamp, data-stream information associated with the portion of the first data stream, and event information associated with the occurrence of one or more events during the sequence of operations. Moreover, the system provides a notification based on the stored metadata, the notification to facilitate synchronizing the first data stream with a second data stream having a second data type.

First claim

What is claimed is:

1. A method for synchronizing data streams, comprising:
receiving a first data stream having a first data type;
performing a sequence of decoding operations on at least a portion of the first data stream, the decoding operations performed at variable processing rates determined at least in part by operating state of a device that performs the decoding operations, wherein performing the sequence of decoding operations involves generating metadata entries associated with the portion of the first data stream, wherein each metadata entry is generated in response to an occurrence of a decoding event during the performance of the sequence of decoding operations, each metadata entry including a time stamp indicating a time of the occurrence, a data-stream identifier indicating the portion of the first data stream that was decoded during the occurrence, and event information describing the respective decoding event; and
storing each metadata entry; and
providing a notification based on the stored metadata, the notification including information by which the decoded first data stream is synchronized with a second data stream having a second data type.

2. The method of claim 1 , wherein the first data stream includes initial metadata.

3. The method of claim 1 , wherein the first data stream includes audio and the second data stream includes video corresponding to the audio.

4. The method of claim 1 , wherein the first data stream includes data associated with a first type of media and the second data stream includes data associated with a second type of media.

5. The method of claim 1 , wherein the sequence of operations includes decoding audio data.

6. The method of claim 1 , wherein the stored metadata indicates where the portion of the first data stream is located with reference to the sequence of operations.

7. The method of claim 1 , further comprising phase shifting the notification by a time interval, the phase shift being positive or negative.

8. The method of claim 1 , wherein the data-stream information associated with at least the portion of the first data stream includes an address of a memory location where the at least the portion of the first data stream is stored.

9. The method of claim 1 , further comprising:
evaluating multiple decoding events in the stored metadata using pre-defined rules that prioritize the decoding events; and
generating the notification using stored metadata for which an associated decoding event is evaluated as being of highest priority.

10. The method of claim 1 , wherein the notification is provided based on a resizing of a memory allocation associated with at least one operation in the sequence of operations.

11. The method of claim 1 , wherein at least one of the decoding events occurs when the associated portion of the first data stream is consumed at an output at the end of the sequence of operations.

12. The method of claim 1 , wherein at least one of the decoding events occurs upon completion of a given decoding operation in the sequence of decoding operations.

13. The method of claim 1 , wherein at least one of the decoding events occurs upon detection of an end of the first data stream.

14. The method of claim 1 , wherein at least one of the decoding events occurs in response to a time-scale modification affecting at least one decoding operation in the sequence of decoding operations, the time-scale modification occurring during the performing of the sequence of decoding operations.

15. The method of claim 1 , wherein at least one of the decoding events occurs upon a change in current media playback time, the change occurring during the performing of the sequence of decoding operations.

16. The method of claim 15 , wherein the change includes fast forwarding, rewinding or seeking.

17. The method of claim 1 , wherein at least one of the decoding events occurs upon changing an active audio-output device during the performing of the sequence of decoding operations.

18. The method of claim 1 , wherein at least one of the decoding events occurs upon completion of the sequence of decoding operations.

19. The method of claim 1 , wherein at least one of the decoding events occurs in response to an error associated with the first data stream or with at least one operation in the sequence of decoding operations.

20. The method of claim 1 , wherein at least one of the metadata entries is generated in response to an internal event associated with at least one decoding operation in the sequence of decoding operations or an external event associated with an application that consumes processed first data output from the sequence of decoding operations.

21. The method of claim 1 , wherein the notification is provided based on a presence of a given decoding event in a subset of the decoding events in the stored metadata.

22. A computer program product for use in conjunction with a computer system, the computer program product comprising a computer-readable storage medium and a computer-program mechanism embedded therein for synchronizing data streams, the computer-program mechanism including:
instructions for receiving a first data stream having a first data type;
instructions for performing a sequence of decoding operations on at least a portion of the first data stream, the decoding operations performed at variable processing rates determined at least in part by operating state of a device that performs the decoding operations, wherein performing the sequence of decoding operations involves generating metadata entries associated with the portion of the first data stream, wherein each metadata entry is generated in response to an occurrence of a decoding event during the performance of the sequence of decoding operations, each metadata entry including a time stamp indicating a time of the occurrence, a data-stream identifier indicating the portion of the first data stream that was decoded during the occurrence, and event information describing the respective decoding event;
instructions for storing each metadata entry; and
instructions for providing a notification based on the stored metadata, the notification including information by which the decoded first data stream is synchronized with a second data stream having a second data type.

23. A computer system to synchronize data streams, comprising:
a processor;
memory; and
a program module, wherein the program module is stored in the memory and configurable to be executed by the processor, the program module including:
instructions for receiving a first data stream having a first data type,
instructions for performing a sequence of decoding operations on at least a portion of the first data stream, the decoding operations performed at variable processing rates determined at least in part by operating state of the processor as it performs the decoding operations, wherein performing the sequence of decoding operations involves generating metadata entries associated with the portion of the first data stream, wherein each metadata entry is generated in response to an occurrence of a decoding event during the performance of the sequence of decoding operations, each metadata entry including a time stamp indicating a time of the occurrence, a data-stream identifier indicating the portion of the first data stream that was decoded during the occurrence, and event information describing the respective decoding event;
instructions for storing each metadata entry; and

Assignees

• Lindahl Aram
• Guetta Anthony J
• Williams Joseph M
• Timis Dan
• Apple Inc

Inventors

• Lindahl Aram
• Guetta Anthony J
• Williams Joseph M
• Timis Dan

Classifications

• H04N21/442

  Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)} · CPC title

• H04N21/4302

  Content synchronisation processes, e.g. decoder synchronisation · CPC title

• H04N21/242 Primary

  Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} · CPC title

• H04N21/8547

  involving timestamps for synchronizing content · CPC title

• H04N21/4307

  Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen · CPC title