Synchronization of data streams with associated metadata streams using smallest sum of absolute differences between time indices of data events and metadata events

Patent metadata

Field	Value
Publication number	US-9535450-B2
Application number	US-201113184542-A
Country	US
Kind code	B2
Filing date	Jul 17, 2011
Priority date	Jul 17, 2011
Publication date	Jan 3, 2017
Grant date	Jan 3, 2017

Abstract

Synchronizing a data stream with an associated metadata stream by receiving a data stream and a metadata stream having a plurality of metadata events associated with the data stream, identifying within the data stream a plurality of data events, matching each of the data events to one of the metadata events in accordance with a matching criterion, and synchronizing the data stream with the metadata stream by effecting a relative time shift between the metadata stream and the data stream in accordance with a time shift adjustment value that results in the smallest sum of absolute differences between time indices of each matched data event and metadata event.

First claim

What is claimed is:

1. A method for synchronizing a data stream with an associated metadata stream, the method comprising:
receiving a data stream and a metadata stream having a plurality of metadata events associated with the data stream, wherein each of the metadata events is associated with a different metadata event time index;
identifying within the data stream a plurality of data events, wherein each of the data events is associated with a different data event time index;
determining a first set of matching time indices for a first relative time shift between the metadata stream and the data stream in accordance with a first time shift adjustment value by matching the data event time indices of a first plurality of the data events to the metadata event time indices of a first plurality of the metadata events where each matched data event time index of the first plurality of data events is closest in time to the metadata event time index with which the matched data event time index is matched;
calculating a first sum of absolute differences between each matched data event time index in the first set of matching time indices and the metadata event time index which the matched data event time index is matched;
determining a second set of matching time indices for a second relative time shift between the metadata stream and the data stream in accordance with a second time shift adjustment value, where the second time shift adjustment value is different than the first time shift adjustment value, by matching the data event time indices of a second plurality of the data events to the metadata event time indices of a second plurality of the metadata events where each matched data event time index of the second plurality of data events is closest in time to the metadata event time index with which the matched data event time index is matched;
calculating a second sum of absolute differences between each matched data event time index in the second set of matching time indices and the metadata event time index which the matched data event time index is matched; and
synchronizing the data stream with the metadata stream by effecting a relative time shift between the metadata stream and the data stream in accordance with one of the time shift adjustment values that results in the smallest of the sums of absolute differences,
where said receiving, identifying, determining, calculating, and synchronizing steps are embodied in any of
a) computer hardware, and
b) computer software embodied in a non-transitory, computer-readable medium.

2. The method of claim 1 where the identifying step comprises identifying each of the data events as speech.

3. The method of claim 2 where the determining step comprises matching where each of the metadata events indicates a change in speaker.

4. The method of claim 1 where the identifying step comprises identifying each of the data events as speech that begins immediately after a non-speech segment of a predefined minimum length.

5. A system for synchronizing a data stream with an associated metadata stream, the system comprising:
a data stream receiving module configured to receive a data stream;
a metadata stream receiving module configured to receive a metadata stream having a plurality of metadata events associated with said data stream, wherein each of the metadata events is associated with a different metadata event time index;
an event detection module configured to identify within the data stream a plurality of data events, wherein each of the data events is associated with a different data event time index; and
a synchronization module configured to
determine a first set of matching time indices for a first relative time shift between the metadata stream and the data stream in accordance with a first time shift adjustment value by matching the data event time indices of a first plurality of the data events to the metadata event time indices of a first plurality of the metadata events where each matched data event time index of the first plurality of data events is closest in time to the metadata event time index with which the matched data event time index is matched,
calculate a first sum of absolute differences between each matched data event time index in the first set of matching time indices and the metadata event time index which the matched data event time index is matched,
determine a second set of matching time indices for a second relative time shift between the metadata stream and the data stream in accordance with a second time shift adjustment value, where the second time shift adjustment value is different than the first time shift adjustment value, by matching the data event time indices of a second plurality of the data events to the metadata event time indices of a second plurality of the metadata events where each matched data event time index of the second plurality of data events is closest in time to the metadata event time index with which the matched data event time index is matched,
calculate a second sum of absolute differences between each matched data event time index in the second set of matching time indices and the metadata event time index which the matched data event time index is matched, and
synchronize the data stream with the metadata stream by effecting a relative time shift between the metadata stream and the data stream in accordance with one of the time shift adjustment values that results in the smallest of the sums of absolute differences,

where said modules are embodied in any of
a) computer hardware, and
b) computer software embodied in a non-transitory, computer-readable medium.

6. The system of claim 5 where each of the data events represent speech.

7. The system of claim 6 where each of the metadata events indicates a change in speaker.

8. The system of claim 5 where each of the data events represent speech that begins immediately after a non-speech segment of a predefined minimum length.

9. A computer program product for synchronizing a data stream with an associated metadata stream, the computer program product comprising:
a non-transitory computer-readable storage medium; and
computer-readable program code embodied in the computer-readable storage medium, where the computer-readable program code is configured to
receive a data stream and a metadata stream having a plurality of metadata events associated with the data stream, wherein each of the metadata events is associated with a different metadata event time index,
identify within the data stream a plurality of data events, wherein each of the data events is associated with a different data event time index,
determine a first set of matching time indices for a first relative time shift between the metadata stream and the data stream in accordance with a first time shift adjustment value by matching the data event time indices of a first plurality of the data events to the metadata event time indices of a first plurality of the metadata events where each matched data event time index of the first plurality of data events is closest in time to the metadata event time index with which the matched data event time index is matched,
calculate a first sum of absolute differences between each matched data event time index in the first set of matching time indices and the metadata event time index which the matched data event time index is matched,
determine a second set of matching time indices for a second relative time shift between the metadata stream and the data stream in accordance with a second time shift adjustment value, where the second time shift adjustment value is different than the first time shift adjustment value, by matching the data event time indices of a second

Assignees

• Ben-David Shay
• Hazanovich Evgeny
• Mandel Zak
• IBM

Inventors

• Ben-David Shay
• Hazanovich Evgeny
• Mandel Zak

Classifications

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• G06F16/9577

  Optimising the visualization of content, e.g. distillation of HTML documents · CPC title

• G06F1/12 Primary

  Synchronisation of different clock signals {provided by a plurality of clock generators} · CPC title

• G11B15/00

  Driving, starting or stopping record carriers of filamentary or web form; Driving both such record carriers and heads; Guiding such record carriers or containers therefor; Control thereof; Control of operating function (driving or guiding heads G11B3/00 - G11B7/00, G11B21/00) · CPC title

• G06F40/103

  Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) · CPC title