Generating congruous metadata for multimedia

Patent metadata

Field	Value
Publication number	US-10216761-B2
Application number	US-4230608-A
Country	US
Kind code	B2
Filing date	Mar 4, 2008
Priority date	Mar 4, 2008
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

A method of generating congruous metadata is provided. The method includes receiving a similarity measure between at least two multimedia objects. Each multimedia object has associated metadata. If the at least two multimedia objects are similar based on the similarity measure and a similarity threshold, the associated metadata of each of the multimedia objects are compared. Then, based on the comparison of the associated metadata of each of the at least two multimedia objects, the method further includes generating congruous metadata. Metadata may be tags, for example.

First claim

What is claimed is:

1. A method for improving metadata for purposes of improving data retrieval, the method comprising:
receiving, by a multimedia data storage and retrieval system server, a first multimedia object comprising a first set of multimedia data and a first set of user-generated metadata and a second multimedia object comprising a second set of multimedia data and a second set of user-generated metadata;
generating, by the multimedia data storage and retrieval system server, a first feature vector representation of the first set of multimedia data of the first multimedia object and a second feature vector representation of the second set of multimedia data of the second multimedia object;
determining, by the multimedia data storage and retrieval system server, a similarity measure between the first multimedia object and the second multimedia object based on a comparison of the first feature vector representation of the first set of multimedia data of the first multimedia object and the second feature vector representation of the second set of multimedia data of the second multimedia object;
comparing, by the multimedia data storage and retrieval system server, the similarity measure and an adjustable similarity threshold in determining that the similarity measure is greater than the adjustable similarity threshold, the adjustable similarity threshold, used in determining that the first and second multimedia objects are similar in order to automatically generate congruous metadata using the first and second sets of user-generated metadata, is configured for adjustment between being more or less restrictive and is adjustable to be more restrictive in an absence of post-generation user verification of the congruous metadata, the post-generation user verification, if available, indicating whether the automatically-generated congruous metadata applies to both the first and second multimedia objects;
in response to determining that the similarity measure is greater than the adjustable similarity threshold, identifying, by the multimedia data storage and retrieval system server, common associated user-generated metadata of the first multimedia object and the second multimedia object by comparing the first set of user-generated metadata to the second set of user-generated metadata to determine the common associated user-generated metadata of the first set of user-generated metadata and the second set of user generated metadata;
automatically generating, by the multimedia data storage and retrieval system server, the congruous metadata based on the common associated user-generated metadata;
storing, by the multimedia data storage and retrieval system server, the congruous metadata associated with both the first and second multimedia objects in a database; and
retrieving, by the multimedia data storage and retrieval system server in response to a retrieval request received from a client computing device, one or both of the first and second multimedia objects using the congruous metadata generated for the first and second multimedia objects.

2. The method of claim 1 , such that the identifying common associated user-generated metadata is based on identifying user-generated metadata in the first set of user-generated metadata that matches user-generated metadata in the second set of user-generated metadata.

3. The method of claim 1 , such that the similarity threshold is a predetermined value.

4. The method of claim 1 , such that the first set of multimedia data comprises a first set of image data and the second set of multimedia data comprises a second set of image data and the similarity measure is based on a comparison of the first set of image data with the second set of image data.

5. The method of claim 1 , such that the first set of multimedia data comprises a first set of audio data and the second set of multimedia data comprises a second set of audio data and the similarity measure is based on a comparison of the first set of audio data with the second set of audio data.

6. The method of claim 1 , such that the first set of multimedia data comprises a first set of video data and the second set of multimedia data comprises a second set of video data and the similarity measure is based on a comparison of the first set of audio data with the second set of video data.

7. The method of claim 6 , such that the similarity measure is based on a comparison of a video mosaic of the first set of video data with a video mosaic of the second set of video data.

8. The method of claim 1 , such that the similarity measure is determined using probabilistic latent semantic analysis (PLSA).

9. The method of claim 1 , such that the similarity measure is determined using a K-nearest neighbor algorithm in pixel space.

10. The method of claim 1 , such that the similarity measure is determined using a local invariant features algorithm.

11. The method of claim 10 , such that local invariant feature algorithm is a Scale-Invariant Feature Transform (SIFT) algorithm.

12. The method of claim 1 , such that the similarity measure is determined using an algorithm based on distances between color histograms.

13. The method of claim 1 , such that the similarity measure is determined by linear predicted coding (LPC).

14. The method of claim 1 , such that the similarity measure is determined by mel-frequency cepstral coefficients.

15. The method of claim 1 , such that the first set of user-generated metadata and the second set of user-generated metadata comprise tags.

16. Computer-readable non-transitory storage media, having tangibly stored thereon, processor-executable instructions, that when executed by a multimedia data storage and retrieval system server, perform a method for improving metadata for purposes of improving data retrieval, the method comprising:
receiving a first multimedia object comprising a first set of multimedia data and a first set of user-generated metadata and a second multimedia object comprising a second set of multimedia data and a second set of user-generated metadata;
generating a first feature vector representation of the first set of multimedia data of the first multimedia object and a second feature vector representation of the second set of multimedia data of the second multimedia object;
determining a similarity measure between the first multimedia object and the second multimedia object based on a comparison of the first feature vector representation of the first set of multimedia data of the first multimedia object and the second feature vector representation of the second set of multimedia data of the second multimedia object;
comparing the similarity measure and an adjustable similarity threshold in determining that the similarity measure is greater than the adjustable similarity threshold, the adjustable similarity threshold, used in determining that the first and second multimedia objects are similar in order to automatically generate congruous metadata using the first and second sets of user-generated metadata, is configured for adjustment between being more or less restrictive and is adjustable to be more restrictive in an absence of post-generation user verification of the congruous metadata, the post-generation user verification, if available, indicating whether the automatically-generated congruous metadata applies to both the first and second multimedia objects;
in response to determining that the similarity measure is greater than the adjustable similarity threshold, identifying common associated user-generated metadata of the first multimedia object and the second multimedia obj

Assignees

• Slaney Malcolm
• Weinberger Kilian
• Oath Inc

Inventors

• Slaney Malcolm
• Weinberger Kilian

Classifications

• G06F17/30265

  Physics · mapped topic

• G06F17/30817

  Physics · mapped topic

• G06F17/30799

  Physics · mapped topic

• G06F17/30256 Primary

  Physics · mapped topic

• G06F17/30743

  Physics · mapped topic