Methods for identifying relevant metadata for multimedia data of a large-scale matching system

Patent metadata

Field	Value
Publication number	US-9798795-B2
Application number	US-34888809-A
Country	US
Kind code	B2
Filing date	Jan 5, 2009
Priority date	Oct 26, 2005
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

A method for associate metadata to a multimedia content based on finding matches to similar multimedia content. A given input multimedia content is matched to at least another multimedia content with corresponding metadata. Upon determination of a match, the corresponding metadata is used as metadata of the given multimedia content. When a large number of multimedia data is compared a ranked list of metadata is provided. The most appropriate metadata is associated to the given multimedia content based on various criteria. The method can be implemented in any applications which involve large-scale content-based clustering, recognition and classification of multimedia data, such as, content-tracking, video filtering, multimedia taxonomy generation, video fingerprinting, speech-to-text, audio classification, object recognition, video search and any other application requiring content-based signatures generation and matching for large content volumes such as, web and other large-scale databases.

First claim

What we claim is:

1. A method for identifying and associating metadata to input multimedia data elements, comprising:
comparing an input first multimedia data element to at least one second multimedia data element by generating for each of said input first multimedia data element and said at least one second multimedia data element a first signature and a second signature, said first signature is a robust signature, and comparing between at least a first signature of said first multimedia data element and at least one of a first signature and a second signature of said at least one second multimedia data element;
collecting metadata of said at least one second multimedia data element when a match is found between said first multimedia data element and said at least one second multimedia data element;
associating, when said match is found between said first multimedia data element and said at least one second multimedia data element, at least a subset of said collected metadata of said at least one second multimedia data element with said first multimedia data element; and
storing said first multimedia data element and said associated metadata in a storage.

2. The method of claim 1 , wherein said at least one second multimedia data element comprises a set of multimedia data elements.

3. The method of claim 1 , wherein a multimedia data element is selected from the group consisting of an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, combinations thereof, and portions thereof.

4. The method of claim 3 , wherein said images of signals are images selected from the group consisting of medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

5. The method of claim 1 , further comprising:
comparing between at least a second signature of said first multimedia data element and at least one of a first signature and a second signature of said at least one second multimedia data element.

6. The method of claim 1 , wherein said comparing is performed on a plurality of computational cores, which each of said computation cores is configured to receive said first multimedia data element.

7. The method of claim 6 , further comprising:
setting properties of each of said plurality of computational cores to be independently of each other core.

8. The method of claim 1 , wherein associating said at least subset of said collected metadata to said first multimedia data element further comprising:
sorting said collected metadata; and
associating a subset of components of said collected metadata that is above a predetermined threshold as metadata for said first multimedia data element.

9. The method of claim 1 , further comprising:
selecting a first metadata from said at least a subset of said collected metadata;
searching for at least a third multimedia data element having said first metadata;
comparing said first multimedia element to said third multimedia element;
collecting metadata associated with said third multimedia data element if a match is found between said first multimedia data element and said third multimedia data element; and
adding said metadata associated with said third multimedia data element to metadata of said first multimedia data element.

10. The method of claim 9 , wherein comparing said first multimedia element to said third multimedia element further comprising:
generating for said third multimedia data element a first signature and a second signature, said first signature is a robust signature; and
comparing between at least a first signature of said first multimedia data element and at least one of a first signature and a second signature of said third multimedia data element.

11. The method of claim 9 , wherein comparing said first multimedia element to said third multimedia element further comprising:
comparing between at least a second signature of said first multimedia data element and at least one of: a first signature and a second signature of said third multimedia data element.

12. A system for collecting metadata for a first multimedia data element, comprising:
a plurality of computational cores configured to receive the first multimedia data element, each core having properties to be independent of each other core, each said core generates responsive to the first multimedia data element a first signature element and a second signature element, said first signature element is a robust signature;
a storage unit for storing at least one second multimedia data element, metadata associated with said second multimedia data element, and at least one of a first signature and a second signature associated with said second multimedia data element, said first signature is a robust signature; and
a comparison unit coupled to said plurality of computational cores and to said storage unit, wherein the comparison unit is configured to compare signatures of multimedia data elements for the purpose of determining matches between multimedia data elements;
wherein responsive to receiving the first multimedia data element said plurality of computational cores generate a respective first signature of said first multimedia element and a second signature of said first multimedia data element, for the purpose of determining a match with said at least one second multimedia data element stored in said storage and associating at least a subset of metadata associated with said at least one second multimedia data element with said first multimedia data element.

13. The system of claim 12 , wherein said multimedia data element is selected from the group consisting of an image, graphics, a video stream, a video clip, an audio stream, an audio clip, a video frame, a photograph, images of signals, a combinations thereof, and portions thereof.

14. The system of claim 12 , wherein said images of signals are images selected from the group consisting of medical signals, geophysical signals, subsonic signals, supersonic signals, electromagnetic signals, and infrared signals.

15. The system of claim 12 , wherein said comparison unit compares between at least a first signature of said first multimedia data element and at least one of: a first signature and a second signature of said at least one second multimedia data element.

16. The system of claim 12 , wherein said comparison unit compares between at least the second signature of said first multimedia data element and at least one of: a first signature and a second signature of said at least one second multimedia data element.

17. The system of claim 12 , said system configured to sort metadata collected from said at least one second multimedia data element for using a subset of said collected metadata that is above a predetermined threshold as metadata for said first multimedia data element.

18. The system of claim 12 , said system is further configured to select a first metadata from said at least a subset of said collected metadata, search for at least a third multimedia data element having said first metadata, compare said first multimedia element to said third multimedia element, collect metadata associated with said third multimedia data element if a match is found between said first multimedia data element and said third multimedia data element, and add said metadata associated with said third multimedia data element to metadata of said first multimedia data element.

19. The system of claim 18 , wherein said system is configured to compare s

Assignees

• Raichelgauz Igal
• Odinaev Karina
• Zeevi Yehoshua Y
• Cortica Ltd

Inventors

• Raichelgauz Igal
• Odinaev Karina
• Zeevi Yehoshua Y

Classifications

• G06F16/2228

  Indexing structures · CPC title

• G06F16/1748

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F16/48

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06V20/48

  Matching video sequences · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title