Topical-based media content summarization system and method

Patent metadata

Field	Value
Publication number	US-10769208-B2
Application number	US-201816151588-A
Country	US
Kind code	B2
Filing date	Oct 4, 2018
Priority date	Apr 9, 2015
Publication date	Sep 8, 2020
Grant date	Sep 8, 2020

Abstract

Disclosed herein is an automated approach for summarizing media content using descriptive information associated with the media content. For example and without limitation, the descriptive information may comprise a title associated with the media content. One or more segments of the media content may be identified to form a media content summary based on each segment's respective similarity to the descriptive information, which respective similarity may be determined using a media content and auxiliary data feature spaces. A shared dictionary of canonical patterns generated using the media content and auxiliary data feature spaces may be used in determining a media content segment's similarity to the descriptive information.

First claim

The invention claimed is:

1. A method comprising:
obtaining, by a digital content summarization system server and for a first media content item having associated descriptive information, a plurality of second media content items as auxiliary data, the obtaining using the descriptive information associated with the first media content item, the first media content item comprising a plurality of units;
identifying, by the digital content summarization system server, a plurality of segments of the first media content item, each segment comprising at least one unit of the first media content item's plurality of units;
generating, by the digital content summarization system server and using the first media content item, a plurality of first feature descriptor sets, each first feature descriptor set of the plurality corresponding to a segment of the plurality of segments and comprising values determined using the segment;
identifying, by the digital content summarization system server, a first set of patterns from the plurality of first feature descriptor sets;
generating, by the digital content summarization system server, a plurality of second feature descriptor sets, each second feature descriptor set, of the plurality, corresponding to a second media content item, of the plurality, and comprising values determined using the second media content item;
identifying, by the digital content summarization system server, a second set of patterns from the plurality of second feature descriptor sets;
determining, by the digital content summarization system server and using the first and second sets of patterns, a shared dictionary comprising a number of shared patterns found in both the first and second sets of patterns;
determining, by the digital content summarization system server and using the shared dictionary, a score for each segment of the plurality of segments of the first media content item, the score determined for a segment, of the plurality of segments, is a measure of how representative the segment is of the first media content item;
identifying, by the digital content summarization system server, a number of segments of the plurality of segments of the first media content item, each segment of the number being selected using its determined score; and
generating, by the digital content summarization system server, a summary of the first media content item, the summary comprising the at least one segment, of the plurality of segments of the first media content item, identified as being more similar to the descriptive information.

2. The method of claim 1 , determining a shared dictionary further comprising:
determining a first approximation coefficient for use in approximating the shared patterns of the shared dictionary, the first approximation coefficient comprising a first approximation coefficient vector for each unit of the plurality of units of the first media content item; and
determining a second approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the first media content item, the second approximation coefficient comprising a second approximation coefficient vector for each unit of the plurality of units of the first media content item.

3. The method of claim 2 , determining a score for a segment of the plurality of segments of the first media content item further comprising:
determining a score for each unit of the segment, the score for a unit of the segment being determined using the first approximation coefficient vector corresponding to the unit and the second approximation coefficient vector corresponding to each unit of the plurality of units; and
determining a score for the segment using the score determined for each unit of the segment.

4. The method of claim 3 , the score determined for the segment is an average determined using the score determined for each unit of the segment.

5. The method of claim 2 , further comprising:
determining a third approximation coefficient for use with the first approximation coefficient, the first media content item and the plurality of second media content items in approximating the shared patterns of the shared dictionary.

6. The method of claim 2 , further comprising:
determining a fourth approximation coefficient that, in combination with the shared patterns of the shared dictionary, approximates the plurality of second media content items.

7. The method of claim 1 , the first media content item is video content and each second media content item of the plurality of second media content items is image content.

8. The method of claim 1 , a shared pattern, of the number of shared patterns, corresponding to values of a common feature descriptor set found in both the first and second feature descriptor sets.

9. The method of claim 1 , the shared dictionary excluding any patterns not found in both the first media content item and the auxiliary data.

10. The method of claim 1 , identifying the second set of patterns from the second set of features further comprising:
extracting, from the second feature descriptor sets, a number of common feature descriptor sets, each common feature descriptor set of the number being found in each of the second feature descriptor sets; and
using the set of common feature descriptor sets in identifying the second set of patterns.

11. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform a method comprising:
obtaining, for a first media content item having associated descriptive information, a plurality of second media content items as auxiliary data, the obtaining using the descriptive information associated with the first media content item, the first media content item comprising a plurality of units;
identifying a plurality of segments of the first media content item, each segment comprising at least one unit of the first media content item's plurality of units;
generating, using the first media content item, a plurality of first feature descriptor sets, each first feature descriptor set of the plurality corresponding to a segment of the plurality of segments and comprising values determined using the segment;
identifying a first set of patterns from the plurality of first feature descriptor sets;
generating a plurality of second feature descriptor sets, each second feature descriptor set, of the plurality, corresponding to a second media content item, of the plurality, and comprising values determined using the second media content item;
identifying a second set of patterns from the plurality of second feature descriptor sets;
determining, using the first and second sets of patterns, a shared dictionary comprising a number of shared patterns found in both the first and second sets of patterns;
determining, using the shared dictionary, a score for each segment of the plurality of segments of the first media content item, the score determined for a segment, of the plurality of segments, is a measure of how representative the segment is of the first media content item;
identifying a number of segments of the plurality of segments of the first media content item, each segment of the number being selected using its determined score; and
generating a summary of the first media content item, the summary comprising the at least one segment, of the plurality of segments of the first media content item, identified as being more similar to the descriptive information.

12. The non-transitory computer-readable storage medium of claim 11 , determining a shared dictionary further comprising:

Assignees

• Oath Inc

Inventors

• Song Yale
• Vallmitjana Jordi
• Stent Amanda
• Jaimes Alejandro

Classifications

• G06F16/739 Primary

  in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames · CPC title

• G06F16/345

  Summarisation for human users · CPC title

• G06F16/78

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F16/38

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title