Apparatus and methods for indexing multimedia content

Patent metadata

Field	Value
Publication number	US-9846696-B2
Application number	US-201213408448-A
Country	US
Kind code	B2
Filing date	Feb 29, 2012
Priority date	Feb 29, 2012
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

A method, medium, and apparatus are disclosed for indexing multimedia content by a computer. The method comprises segmenting the multimedia content into a plurality of segments. For each segment, the method identifies one or more features present in the segment, wherein the features are of respective media types. The method then identifies, for each identified feature in each segment, one or more respective keywords associated the identified feature. Then, the method determines, for each identified keyword associated with an identified feature in a given segment, a respective relevance of the keyword to the given segment. The respective relevance is dependent on a weight associated with the respective media type of the identified feature.

First claim

The invention claimed is:

1. A method for indexing multimedia content, the method comprising:
segmenting the multimedia content, by a computer, into a plurality of segments;
identifying by the computer, for each segment, one or more features present in the segment, wherein the features are of respective media types;
identifying by the computer, for each identified feature in each segment, one or more respective keywords associated the identified feature; and
determining by the computer, for each identified keyword associated with an identified feature in a given segment, a respective relevance of the keyword to the given segment, wherein the respective relevance is dependent on a weight associated with the respective media type of the identified feature.

2. The method of claim 1 , further comprising:
creating, by the computer, an index for the multimedia content, wherein the index correlates each identified keyword with those of the plurality of segments that are determined as being relevant to the keyword based on the determined respective relevances;
receiving, by the computer, a query for multimedia content;
determining, by a the computer, a keyword based on the query;
determining, by a the computer, a segment of the plurality of segments based on the determined keyword and the index; and
providing, by the computer, an indication of the segment.

3. The method of claim 2 , further comprising: providing, by the computer, in response to the query, a list of segments arranged in order of relevance.

4. The method of claim 1 , further comprising: determining a relevance of at least one of the keywords to a file comprising multiple segments, wherein the keyword's relevance to the file is dependent on the relevances of the keyword to the multiple segments of the file.

5. The method of claim 1 , wherein the respective relevance of the keyword to the given segment is further dependent on a prominence of the identified feature associated with the keyword within the given segment.

6. The method of claim 5 , wherein the prominence is dependent on a proportion of the given segment in which the identified feature is present.

7. The method of claim 1 , wherein the content comprises a plurality of audio-visual files.

8. The method of claim 1 , wherein the different media types include at least one of image, sound, text, speech, or content metadata.

9. The method of claim 8 , wherein the content metadata includes at least one of: geographic positioning data, user generated text associated with the feature, or other user generated tags.

10. A non-transitory computer-readable storage medium storing program instructions executable by a computer to implement a multimedia content indexer configured to:
segment multimedia content into a plurality of segments;
identify, for each segment, one or more features present in the segment, wherein the features are of respective media types;
identify, for each identified feature in each segment, one or more respective keywords associated the identified feature; and
determine, for each identified keyword associated with an identified feature in a given segment, a respective relevance of the keyword to the given segment, wherein the respective relevance is dependent on a weight associated with the respective media type of the identified feature.

11. The medium of claim 10 , wherein the program instructions are further executable to:
create an index for the multimedia content, wherein the index correlates each identified keyword with those of the plurality of segments that are determined as being relevant to the keyword based on the determined respective relevances.

12. The medium of claim 11 , wherein the program instructions are further executable to:
receive a query for multimedia content;
determine a keyword based on the query;
determine a segment of the plurality of segments based on the determined keyword and the index; and
provide an indication of the segment.

13. The medium of claim 10 , wherein the program instructions are further executable to: determine a relevance of at least one of the keywords to a file comprising multiple segments, wherein the keyword's relevance to the file is dependent on the relevances of the keyword to the multiple segments of the file.

14. The medium of claim 10 , wherein the respective relevance of the keyword to the given segment is further dependent on a prominence of the identified feature associated with the keyword within the given segment.

15. The medium of claim 10 , wherein the different media types include at least one of image, sound, text, speech, or content metadata.

16. An apparatus comprising:
a processor;
a memory coupled to the processor, wherein the memory stores program instructions executable by the processor to implement a multimedia content indexer configured to:
segment multimedia content into a plurality of segments;
identify, for each segment, one or more features present in the segment, wherein the features are of respective media types;
identify, for each identified feature in each segment, one or more respective keywords associated the identified feature; and
determine, for each identified keyword associated with an identified feature in a given segment, a respective relevance of the keyword to the given segment, wherein the respective relevance is dependent on a weight associated with the respective media type of the identified feature.

17. The apparatus of claim 16 , wherein the program instructions are further executable to:
create an index for the multimedia content, wherein the index correlates each identified keyword with those of the plurality of segments that are determined as being relevant to the keyword based on the determined respective relevances.

18. The apparatus of claim 17 , wherein the program instructions are further executable to:
receive a query for multimedia content;
determine a keyword based on the query;
determine a segment of the plurality of segments based on the determined keyword and the index; and
provide an indication of the segment.

19. The method of claim 1 , wherein
a first of said one or more respective keywords is associated with a first identified feature and a second identified feature, and
the step of determining the respective relevance of said first keyword comprises:
determining a first feature score for the first identified feature;
determining a second feature score for the second identified feature;
determining a first keyword score for the first keyword, wherein determining the first keyword score for the first keyword comprises calculating the first keyword score using the first feature score for the first identified feature and the second feature score for the second identified feature; and
associating the calculated first keyword score with the first keyword.

20. The method of claim 19 , wherein the step of determining a first feature score for the first identified feature comprises:
determining a respective media type of the first identified feature;
determining a weight value associated with the determined respective media type;
determining a prominence value identifying a prominence of the first identified feature within the segment in which the first identified features was identified; and
calculating the first feature score using the weight value and the prominence value.

21. The method of claim 1 , wherein
a given one of the identified keywords is associated with, at the least, i) a first feature presen

Assignees

• Arngren Tommy
• Soderberg Joakim
• Stalnacke Marika
• ERICSSON TELEFON AB L M (publ)

Inventors

• Arngren Tommy
• Soderberg Joakim
• Stalnacke Marika

Classifications

• G06F16/41 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/7867

  using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings · CPC title

• G06F16/71

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F17/30858

  Physics · mapped topic

• G06F17/3082

  Physics · mapped topic