Methods and systems for aggregation and organization of multimedia data acquired from a plurality of sources

Patent metadata

Field	Value
Publication number	US-10277946-B2
Application number	US-201715699446-A
Country	US
Kind code	B2
Filing date	Sep 8, 2017
Priority date	Oct 10, 2013
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

A system for aggregation and organization of multimedia data acquired from a plurality of sources can include a data collection element, a multimedia segmentation element, a memory, an extraction element, and a user interface. The data collection element can be configured to receive multimedia data. The multimedia segmentation element can be coupled to the data collection element and can be configured to separate the multimedia data into a plurality of multimedia data segments. The memory can be configured to store hierarchical category information having a plurality of categories. The extraction element can be coupled to the multimedia segmentation element and the memory and can be configured to associate at least one of the plurality of categories with each of the plurality of multimedia data segments.

First claim

The invention claimed is:

1. A system for linking one or more segments of multimedia data to social media, comprising:
a processor configured to extract hierarchical category information by:
identifying at least one text region in each of the one or more segments, if any, containing text and extracting the text from the at least one text region in each of the one or more segments, if any, or
identifying a speaker based on a detected face track, applying a detected name in on-screen text and applying that to the speaker, or applying a detected name in an audio transcript based on temporal proximity to the speaker in each of the one or more segments, if any;

a memory, coupled to the processor, and configured to store the hierarchical category information based on the extracted text or the identified speaker; and
the processor further configured for:
associating the extracted hierarchical category information with the one or more segments of multimedia data, and
linking the one or more associated segments of multimedia data to social media based on a social media identifier.

2. The system of claim 1 , wherein the social media identifier is a hashtag.

3. The system of claim 1 , wherein a representation of the social media identifier is constructed by aggregating content tagged with the social media identifier.

4. The system of claim 3 , wherein a similarity metric is generated that compares the representation of the social media identifier to the one or more associated segments of multimedia data.

5. The system of claim 4 , wherein when the similarity metric exceeds a predefined threshold, the social medial identifier is linked the one or more associated segments of multimedia data.

6. A method for linking one or more segments of multimedia data to social media, comprising:
extracting, by a processor, hierarchical category information by:
identifying at least one text region in each of the one or more segments, if any, containing text and extracting the text from the at least one text region in each of the one or more segments, if any, or
identifying a speaker based on a detected face track, applying a detected name in on-screen text and applying that to the speaker, or applying a detected name in an audio transcript based on temporal proximity to the speaker in each of the one or more segments, if any;

storing, by a memory, the hierarchical category information based on the extracted text or the identified speaker;
associating, by the processor, the extracted hierarchical category information with the one or more segments of multimedia data; and
linking the one or more associated segments of multimedia data to social media based on a social media identifier.

7. The method of claim 6 , wherein the social media identifier is a hashtag.

8. The method of claim 6 , wherein a representation of the social media identifier is constructed by aggregating content tagged with the social media identifier.

9. The method of claim 8 , wherein a similarity metric is generated that compares the representation of the social media identifier to the one or more associated segments of multimedia data.

10. The method of claim 9 , wherein when the similarity metric exceeds a predefined threshold, the social medial identifier is linked the one or more associated segments of multimedia data.

Assignees

• Univ Columbia

Inventors

• Chang Shih-Fu
• Jou Brendan
• Li Hongzhi
• Ellis Joseph G
• Morozoff-Abezgauz Daniel
• Karaman Svebor

Classifications

• H04N21/8126

  involving additional data, e.g. news, sports, stocks, weather forecasts · CPC title

• H04N21/4622 Primary

  Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859) · CPC title

• H04N21/4394

  involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58) · CPC title

• H04N21/858

  Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot · CPC title

• H04N21/4826

  using recommendation lists, e.g. of programmes or channels sorted out according to their score · CPC title