Automatic synchronization of audio and video based media services of media content

Patent metadata

Field	Value
Publication number	US-9628851-B2
Application number	US-54457703-A
Country	US
Kind code	B2
Filing date	Feb 14, 2003
Priority date	Feb 14, 2003
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

There is provided a method for automatically synchronizing playback between a first media service and a second media service, both services corresponding to the same media content source. Common audio events in the first media service and the second media service are matched. A playback timing of at least one of the first media service and the second media service is adjusted, based on a result of the matching step, so as to automatically synchronize the playback between the first media service and the second media service.

First claim

The invention claimed is:

1. A method for synchronizing playback between a first media service and a second media service, corresponding to a same media content, on a computing device, the method comprising:
matching common audio characteristics in an audio signal of the first media service with corresponding audio characteristics in an audio waveform corresponding to the second media service based on codes embedded in meta data associated with the first media service and the second media service; and
the computing device adjusting a playback timing of at least one of the first media service and the second media service, which is a different media service than the first media service, based on a result of said matching step, so as to synchronize the playback of the first media service and the second media service.

2. The method of claim 1 , wherein said matching step matches the common audio characteristics in the audio signal of first media service and the audio waveform using noise envelope matching.

3. The method of claim 1 , wherein said matching step matches the common audio characteristics in the audio signal of the first media service and the audio waveform using audio fingerprints of known sounds.

4. The method of claim 3 , wherein the known sounds comprise at least one of a stadium charge sound, a scoring sound, and a song.

5. The method of claim 1 , wherein said matching step comprises the step of selecting a time window for sampling the audio signal of the first media service and the audio wave form corresponding to the second media service, based on a worst case expected delay difference between the first media service and the second media service.

6. The method of claim 1 , wherein said matching step comprises the step of generating a spectral power density curve for both the audio signal of the first media service and the audio waveform.

7. The method of claim 1 , wherein the common audio characteristics are background audio events and said matching step comprises the step of correlating the background audio events occurring in the audio signal of the first media service and the audio waveform.

8. The method of claim 1 , wherein said matching step comprises the step of minimizing a correlation error between spectral power density curves corresponding to the audio signal of the first media service and audio waveform.

9. The method of claim 1 , wherein said matching step comprises:
selecting a time window for sampling the audio signal of the first media service and an audio waveform corresponding to the second media service;
generating spectral power density curves for the audio signal and the audio waveform;
correlating background audio events occurring in the spectral power density curves for the audio signal and the audio waveform; and
minimizing a correlation error between the spectral power density curves corresponding to the audio signal and the audio waveform.

10. The method of claim 1 , wherein said matching step comprises:
comparing audio fingerprints of known sounds to sounds in audio signal of the first media service and the audio waveform corresponding to the second media service; and
correlating sounds that match an audio fingerprint and that occur in both the audio signal and the audio waveform.

11. An apparatus for synchronizing a playback between a first media service and a second media service corresponding to a same media content, the apparatus comprising:
a sound-matching device for matching common audio characteristics in an audio signal of the first media service and in an audio waveform corresponding to the second media service, which is different than the first media service, said matching being performed based on codes embedded in meta data associated with the first media service and the second media service; and
a variable delayer for adjusting a playback timing of at least one of the first media service and the second media service, based on an output of said sound-matching device, so as to synchronize the playback between the first media service and the second media service.

12. The apparatus of claim 1 , wherein said sound-matching device matches the common audio events in the audio signal of first media service and the audio waveform using noise envelope matching.

13. The apparatus of claim 1 , wherein said sound-matching device matches the common audio events in the audio signal of the first media service and the audio waveform using audio fingerprints of known sounds.

14. The apparatus of claim 1 , wherein said sound-matching device selects a time window for sampling the audio signal of the first media service and the audio waveform, based on a worst case expected delay difference between the audio signal and the audio waveform.

15. The apparatus of claim 1 , wherein said sound-matching device generates a spectral power density curve for both the audio signal of the first media service and the audio waveform.

16. The apparatus of claim 1 , wherein said sound-matching device minimizes a correlation error between spectral power density curves corresponding to the audio signal of the first media service and audio waveform.

17. The apparatus of claim 1 , wherein said sound-matching device selects a time window for sampling the audio signal of the first media service and the audio waveform corresponding to the second media service, generates spectral power density curves for the audio signal and the audio waveform, correlates background audio events occurring in the spectral power density curves for the audio signal and the audio waveform, and minimizes a correlation error between the spectral power density curves corresponding to the audio signal and the audio waveform.

18. The apparatus of claim 1 , wherein said sound-matching device compares audio fingerprints of known sounds to the audio characteristics in the audio signal of the first media service and the audio waveform, and correlates sounds that match an audio fingerprint and that occur in both the audio signal and the audio waveform.

19. The apparatus of claim 1 , wherein the first media content is of a first authoring source and the second media content is of a second authoring source.

Assignees

• Hu Carl Chun-Hei
• Strothmann James Alan
• Thomson Licensing

Inventors

• Hu Carl Chun-Hei
• Strothmann James Alan

Classifications

• H04N21/4341

  Demultiplexing of audio and video streams · CPC title

• H04N5/04

  Synchronising (for television systems using pulse code modulation H04N7/56) · CPC title

• H04N21/443

  OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB · CPC title

• H04N21/4394 Primary

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N21/4622

  Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859) · CPC title