Systems and methods to associate multimedia tags with user comments and generate user modifiable snippets around a tag time for efficient storage and sharing of tagged items

Patent metadata

Field	Value
Publication number	US-9966112-B1
Application number	US-201414255742-A
Country	US
Kind code	B1
Filing date	Apr 17, 2014
Priority date	Apr 18, 2013
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

In order to organize and reference multimedia content that is presented on a television or other media devices, a tagging system and method are utilized. An approach is described to tag multimedia content at specific times, record text, audio and/or video, and comment at specific times in the multimedia content. For tagging, commenting, and sharing particular moments of multimedia content, automatic content recognition (ACR) is used. ACR supports tagging and recording of snippets of the multimedia content at specific times. Snippets are displayed by using a thumbnail of pictures or small multimedia clips. The snippets can be commented on and shared with selected users or groups of users. An automatic highlight playlist of a multimedia content can be generated, and various filtering operations of the tags, comments and snippets can also be performed.

First claim

We claim:

1. A method to add a tagged snippet of multimedia content to a playlist based on a number of viewers, the method comprising:
searching a fingerprint database to find a match between query fingerprints for a snippet of multimedia content captured from a multimedia program at a timestamp and original fingerprints of matching original multimedia content stored in the fingerprint database that identifies the snippet of multimedia content and the multimedia program;
generating a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with the matching original multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a database storage as a tagged snippet of multimedia content; and
adding the tagged snippet of multimedia content to a playlist for the identified multimedia program if the number of viewers of the tagged snippet exceeds a threshold.

2. The method of claim 1 further comprising:
generating additional query fingerprints to improve identification of the captured snippet of multimedia content; and
supplying the generated additional query fingerprints to be stored with the tag.

3. The method of claim 1 further comprising:
running an automatic content recognition (ACR) process to search for a match between the generated query fingerprints and the original fingerprints stored in the fingerprint database; and
retrieving the meta information associated with the matching original multimedia content.

4. The method of claim 1 further comprising:
generating a snippet with an associated tag representing the matching original multimedia content provided by an automatic content recognition (ACR) analysis in a search system; and
storing the generated snippet in a snippet storage database.

5. The method of claim 4 further comprising:
accessing the snippet from the snippet storage database; and
displaying a plurality of playlists including a playlist of one tag representing the accessed snippet, wherein the user selects the playlist of the one tag to initiate playing of the accessed snippet.

6. The method of claim 1 further comprising:
storing a plurality of snippets from the identified multimedia program according to a specified duration, wherein each snippet from the identified multimedia program is a selectively captured snippet of multimedia content having an associated tag; and
displaying a playlist of tags representing a sequence of selectively captured segments of multimedia content corresponding to a sequence of snippets, wherein the user selects the playlist to initiate playing of the sequence of the selectively captured snippets.

7. The method of claim 1 further comprising:
associating a current time with the identified multimedia program that is known to be playing at the current time and at a channel selected by a user, wherein the matching original multimedia content found in the database is identified in the meta information that includes scheduled playing time and channel.

8. The method of claim 1 further comprising:
requesting identification of the captured snippet of multimedia content associated with the tag; and
responding with a user identification for the captured snippet of multimedia content in the meta information associated with the tag.

9. The method of claim 1 further comprising:
sharing the tag with users in a user group specified in sharing primitives stored with the tag.

10. The method of claim 1 further comprising:
repeating the fingerprinting, searching, and generating processes for each captured segment of multimedia content to create a playlist of tags representing a sequence of the selectively captured segments of multimedia content.

11. The method of claim 1 ,
wherein the threshold is a fixed threshold that does not change as a number of viewers changes during repeated playing of the identified selected multimedia program.

12. The computer implemented method of claim 1 ,
wherein the threshold is a dynamic threshold that is a percentage of a number of viewers, and wherein the number of viewers that exceed the dynamic threshold changes during repeated playing of the identified selected multimedia program.

13. The method of claim 1 , wherein a collection of fingerprints drawn from a single media frame with one or more features generates a compact media frame representation and a collection of fingerprints associated with each media frame in a sequence with multiple frames forms a media sequence representation.

14. The method of claim 1 further comprising:
capturing the snippet of multimedia content from the selected multimedia program at the timestamp;
extracting media features from a frame of the captured snippet of multimedia content; and
converting the extracted media features into the query fingerprints.

15. A computer implemented method of automatically editing a playlist of snippets of multimedia content based on a number of viewers, the method comprising:
selectively capturing a snippet of multimedia content from a multimedia program at a timestamp on a mobile device;
generating a tag representing the snippet of multimedia content, wherein the tag, the timestamp, meta information associated with matching original multimedia content, and a monitored variable for a number of viewers of the snippet of multimedia content are stored in a snippet database as a tagged snippet of multimedia content;
comparing the tagged snippet of multimedia content at a selected insertion point in a playlist of tags with other snippets in the playlist of tags if the number of viewers of the tagged snippet exceeds a threshold to automatically determine whether there is an overlap between the tagged snippet and the other snippets; and
automatically removing duplicate content from the tagged snippet determined to have the overlap between the tagged snippet and the other snippets to create a modified snippet with an adjusted size without overlap for insertion in the playlist of tags.

16. The computer implemented method of claim 15 further comprising:
moving an initial start point to a final start point for overlap determined at the beginning of the identified snippet.

17. The computer implemented method of claim 15 further comprising:
moving the modified snippet to a new position in the playlist of tags, to generate a modified playlist of tags.

18. The computer implemented method of claim 15 further comprising:
moving an initial stop point to a final stop point for overlap determined at the end of the identified snippet.

19. A computer implemented method of automatically generating a playlist of snippets of multimedia content, the method comprising:
receiving multimedia content on a user device;
generating a first plurality of snippets from the received multimedia content according to a specified parameter;
selecting a second plurality of snippets from the first plurality of snippets according to a predefined threshold;
ordering the second plurality of snippets in a playlist according to a start time of each snippet in the ordered playlist; and
processing snippets from the ordered playlist that are overlapping to automatically remove duplication of multimedia content from the ordered playlist and generate a final playlist with non-overlapping snippets.

20. The computer implemented method of claim 19 further comprising:
adding a new user selected snippet to the final playlist to create a modified playlist;
reordering

Assignees

• Gracenote Inc

Inventors

• Kulkarni Sunil Suresh
• Bolgarov Oleksiy

Classifications

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• G11B27/031

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title

• H04N21/26258 Primary

  for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list (retrieval of multimedia data based on playlists G06F16/40) · CPC title

• H04N21/4788

  communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} · CPC title

• H04N21/458

  Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} · CPC title