Context-based analysis of multimedia content items using signatures of multimedia elements and matching concepts

Patent metadata

Field	Value
Publication number	US-9639532-B2
Application number	US-201314096901-A
Country	US
Kind code	B2
Filing date	Dec 4, 2013
Priority date	Oct 26, 2005
Publication date	May 2, 2017
Grant date	May 2, 2017

Abstract

A method and server for analyzing a multimedia content item are provided. The method comprises receiving a multimedia content item; extracting from the multimedia content item a plurality of multimedia elements; generating at least one signature for each of the plurality of multimedia elements; for each of the plurality of multimedia elements, querying a deep-content-classification (DCC) system to identify at least one concept that matches one of the plurality of multimedia elements, wherein querying is performed using the at least one signature generated for the multimedia elements and wherein an unidentified multimedia content element does not have a matching concept; generating a context for the multimedia content item using matching concepts; and characterizing each unidentified multimedia element using the generating context and signatures of the matching concepts.

First claim

What is claimed is:

1. A method for analyzing multimedia content items, comprising:
receiving a multimedia content item;
extracting from the multimedia content item a plurality of multimedia elements;
generating at least one signature for each of the plurality of multimedia elements;
querying a deep-content-classification (DCC) system to identify at least one concept that matches at least one of the plurality of multimedia elements, wherein querying is performed using the at least one signature generated for the multimedia elements and wherein a multimedia element is unidentified when the multimedia element does not have a matching concept identified via the querying;
generating a context for the multimedia content item using matching concepts; and
characterizing each unidentified multimedia element using the generated context and signatures of the identified at least one concept.

2. The method of claim 1 , wherein the multimedia content item is at least one of: an image, a graphic, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, and images of signals.

3. The method of claim 2 , wherein the images of signals are one of: medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, infrared signals, an audio signal, a video signal, coordinates, and a sonography signal.

4. The method of claim 1 , wherein the at least one signature generated for each of the plurality of multimedia elements is robust to noise and distortions.

5. The method of claim 1 , wherein a matching concept is a collection of signatures representing a multimedia element and metadata describing the concept, wherein the collection is a signature reduced cluster generated by inter-matching signatures generated for the plurality of multimedia elements, wherein the matching concept is represented using at least one signature.

6. The method of claim 1 , wherein a concept is determined to match a multimedia element when the at least one signature of the concept matches at least one signature generated for the multimedia element over a predefined threshold.

7. The method of claim 5 , wherein upon identification of each matching concept the at least one signature representing the concept and metadata matching the concept are returned.

8. The method of claim 7 , wherein characterization of each unidentified multimedia content element is performed in part using metadata of the matching concepts.

9. The method of claim 8 , wherein the characterization of each unidentified multimedia element includes a correlation of the at least one signature of the unidentified multimedia element against signatures of the matching concepts and signatures of the generated context.

10. The method of claim 9 , wherein the correlation of the at least one signature of the unidentified multimedia content element against signatures of the matching concepts further comprising:
identifying any one of a ratio of sizes, angles, scale, location and orientation between the at least one signature of the unidentified multimedia element and signatures of the matching concepts.

11. The method of claim 10 , wherein the correlation of the at least one signature of the unidentified multimedia element against signatures of the matching concepts is performed by at least one probabilistic model.

12. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 1 .

13. A server for analyzing a multimedia content item, comprising:
an interface to a network for receiving a multimedia content item;
a processor; and
a memory connected to the processor, the memory contains instructions that when executed by the processor, configure the server to:
extract from the multimedia content item a plurality of multimedia elements;
generate at least one signature for each of the plurality of multimedia elements;
query a deep-content-classification (DCC) system to identify at least one concept that matches at least one of the plurality of multimedia content elements, wherein querying is performed using the at least one signature generated for the multimedia elements and wherein a multimedia element is unidentified when the multimedia element does not have a matching concept identified via the query;
generate a context for the multimedia content item using the matching concepts; and
characterize each unidentified multimedia element using the generated context and signatures of the identified at least one concept.

14. The server of claim 13 , wherein the multimedia content item is at least one of: an image, a graphic, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, and images of signals.

15. The server of claim 14 , wherein the images of signals are one of: medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, infrared signals, an audio signal, a video signal, coordinates, and a sonography signal.

16. The server of claim 13 , wherein a matching concept is a collection of signatures representing a multimedia element and metadata describing the concept, wherein the collection is a signature reduced cluster generated by inter-matching signatures generated for the plurality of multimedia elements, wherein the matching concept is represented using at least one signature.

17. The server of claim 13 , wherein a concept is determined to match a multimedia element when the at least one signature of the concept matches at least one signature generated for the multimedia element over a predefined threshold.

18. The server of claim 17 , wherein upon identification of each matching concept the at least one signature representing the concept and metadata matching the concept are returned.

19. The server of claim 18 , wherein characterization of each unidentified multimedia element is performed in part using metadata of matching concepts.

20. The server of claim 19 , wherein the characterization of each unidentified multimedia element includes a correlation of the at least one signature of the unidentified multimedia element against signatures of the matching concepts and signatures of the generated context.

21. The server of claim 20 , wherein the correlation of the at least one signature of the unidentified multimedia element against signatures of the matching concepts further comprises identifying any one of a ratio of sizes, angles, scale, location and orientation between the at least one signature of the unidentified multimedia element and signatures of the matching concepts.

22. The server of claim 21 , wherein the correlation of the at least one signature of the unidentified multimedia element against signatures of the matching concepts is performed by at least one probabilistic model.

Assignees

• Cortica Ltd

Inventors

• Raichelgauz Igal
• Odinaev Karina
• Zeevi Yehoshua Y

Classifications

• H04N21/466

  Learning process for intelligent management, e.g. learning user preferences for recommending movies {(services using the results of monitoring in broadcast systems H04H60/61)} · CPC title

• H04H60/64

  for providing detail information · CPC title

• H04H60/73

  using meta-information · CPC title

• H04N21/8106

  involving special audio data, e.g. different tracks for different languages · CPC title

• H04N21/25891

  being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18) · CPC title