Topical based media content summarization system and method

Patent metadata

Field	Value
Publication number	US-2016299968-A1
Application number	US-201514682654-A
Country	US
Kind code	A1
Filing date	Apr 9, 2015
Priority date	Apr 9, 2015
Publication date	Oct 13, 2016
Grant date	—

Abstract

Disclosed herein is an automated approach for summarizing media content using descriptive information associated with the media content. For example and without limitation, the descriptive information may comprise a title associated with the media content. One or more segments of the media content may be identified to form a media content summary based on each segment's respective similarity to the descriptive information, which respective similarity may be determined using a media content and auxiliary data feature spaces. A shared dictionary of canonical patterns generated using the media content and auxiliary data feature spaces may be used in determining a media content segment's similarity to the descriptive information.

First claim

1 . A method comprising:
obtaining, using at least one computing device, a plurality of items of auxiliary data using descriptive information associated with a media content item, the media content item comprising a plurality of units; generating, using the at least one computing device, a media content item feature space and an auxiliary data feature space; identifying, using the at least one computing device, a plurality of segments of the media content item, each segment comprising at least one unit of the media content item's plurality of units; scoring, using the at least one computing device, each segment of the plurality of segments of the media content items using the media content item feature space and the auxiliary data feature space, each segment's score representing a measure of similarity of the segment to the descriptive information; identifying, using the at least one computing device, at least one segment of the plurality of segments of the media content item as more similar to the descriptive information relative to others of the plurality of segments using the scoring of the plurality of segments; and generating, using the at least one computing device, a media content item summary comprising the at least one segment of the plurality identified as being more similar to the descriptive information.

2 . The method of claim 1 , the descriptive information comprising a title of the media content item.

3 . The method of claim 1 , the plurality of canonical patterns appear in both the media content item and the auxiliary data.

4 . The method of claim 1 , scoring further comprising:
determining, for each segment, a unit-level score for each of the at least one unit of the segment using the media content item feature space and the auxiliary data feature space, each unit level score representing a measure of similarity of a respective one of the at least one unit to the descriptive information; and scoring each segment using the unit-level score determined for each of the at least one unit of the segment.

5 . The method of claim 4 , each segment's score comprising an average unit-level score determined using each unit-level score determined for each of the at least one unit of the segment.

6 . The method of claim 4 , the media content item is a video content item, each unit of the media content item is a frame and each segment comprises at least one frame.

7 . The method of claim 1 , generating a media content item feature space and an auxiliary data feature space further comprising:
generating, by the at least one computing device, the media content item feature space by generating, for each of the plurality of units of the media content item, a plurality of feature descriptor values, each feature descriptor value corresponding to a feature of a set of features; and generating, by the at least one computing device, the auxiliary data feature space by generating a plurality of feature descriptor values for each item of auxiliary data, each feature descriptor value corresponding to a feature of the set of features used in generating the plurality of feature descriptor values for each unit of the media content item.

8 . The method of claim 1 , the scoring further comprising:
determining, using the at least one computing device, the media content item feature space and the auxiliary data feature space, a shared dictionary comprising a plurality of canonical patterns shared by the media content item and the auxiliary data; and scoring, using the at least one computing device, the plurality of segments of the media content item, for each segment of the plurality of segments, the scoring comprising determining a measure of similarity of the segment to the descriptive information using the shared dictionary.

9 . The method of claim 8 , the determining further comprising:
determining a first set of coefficients for use with the shared dictionary in approximating the plurality of feature descriptor values of each unit of the plurality of units of the media content item; determining a second set of coefficients for use with the shared dictionary in approximating the plurality of feature descriptor values of each item of the auxiliary data; and determining third and fourth sets of coefficients, the third set of coefficients for use with the plurality of feature descriptor values of each unit of the plurality of units and the fourth set of coefficients for use with the plurality of feature descriptor values of each item of the plurality of items of auxiliary data in approximating the shared dictionary.

10 . The method of claim 9 , the scoring further comprising:
scoring each unit of the plurality of units of media content item, the scoring comprising determining a unit-level score for each unit representing the unit's measure of similarity to the media content item's descriptive information using a plurality of coefficients from the first and third sets of coefficient.

11 . The method of claim 8 , the determining further comprising:
learning the shared dictionary's canonical patterns such that each unit of the plurality of units of the media content item and each item of the plurality of items of the auxiliary data is independently approximated by a combination of the plurality of canonical patterns of the shared dictionary and such that each canonical pattern of the plurality of canonical patterns of the shared dictionary is jointly approximated by a combination of the plurality of units of the media content item and the plurality of items of auxiliary data.

12 . The method of claim 11 , each unit of the media content item and each item of the plurality of items of the auxiliary data is independently approximated by a convex combination of the plurality of canonical patterns of the shared dictionary and each canonical pattern of the plurality of canonical patterns of the shared dictionary is jointly approximated by a convex combination of the plurality of units of the media content item and the plurality of items of auxiliary data.

13 . A system comprising:
at least one computing device, each computing device comprising one or more processors and a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic comprising:
obtaining logic executed by the one or more processors for obtaining a plurality of items of auxiliary data using descriptive information associated with a media content item, the media content item comprising a plurality of units;
generating logic executed by the one or more processors for generating a media content item feature space and an auxiliary data feature space;
identifying logic executed by the one or more processors for identifying a plurality of segments of the media content item, each segment comprising at least one unit of the media content item's plurality of units;
scoring logic executed by the one or more processors for scoring each segment of the plurality of segments of the media content items using the media content item feature space and the auxiliary data feature space, each segment's score representing a measure of similarity of the segment to the descriptive information;
identifying logic executed by the one or more processors for identifying at least one segment of the plurality of segments of the media content item as more similar to the descriptive information relative to others of the plurality of segments using the scoring of the plurality of segments; and
generating logic executed by the one or more processors for generating a media content item summary comprising the at least one segment of the plurality identified as being mor

Assignees

• Yahoo Inc

Inventors

• Song Yale
• Vallmitjana Jordi
• Stent Amanda
• Jaimes Alejandro

Classifications

• G06F16/38

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F16/739 Primary

  in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames · CPC title

• G06F16/345

  Summarisation for human users · CPC title

• G06F16/78

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F17/30719 Primary

  Physics · mapped topic