Retrieving visual media

Patent metadata

Field	Value
Publication number	US-9229958-B2
Application number	US-201114239387-A
Country	US
Kind code	B2
Filing date	Sep 27, 2011
Priority date	Sep 27, 2011
Publication date	Jan 5, 2016
Grant date	Jan 5, 2016

Abstract

Examples of the present disclosure may include methods, systems, and computer readable media with executable instructions. An example method for retrieving visual media can include receiving a text query associated with a target content. A first group of visual media is identified based on correspondence of metadata of the visual media with the text query, and keyframes from the first group of identified visual media are selected. The method further includes detecting instances of a content type in the selected keyframes, and grouping similar instances of the content type into clusters. The target content is associated with a cluster having a greatest quantity of similar instances.

First claim

What is claimed:

1. A method for retrieving visual media, comprising:
receiving, using a processor, a text query associated with a target content;
identifying, using the processor, a first group of visual media based on correspondence of metadata of the visual media with the text query;
selecting, using the processor, keyframes from the first group of identified visual media;
detecting, using the processor, instances of a content type in the selected keyframes;
grouping, using the processor, similar instances of the content type into clusters;
associating, using the processor, the target content with a cluster having a greatest quantity of similar instances;
indexing, using the processor, locations within the visual media based on content type in the selected keyframes; and
repurposing, using the processor, images directly from the indexed locations within the visual media into a new customized visual media product.

2. The method of claim 1 , wherein:
receiving, using the processor, a text query associated with a target content includes receiving a name, the target content being one or more images of a person with the name; and
detecting, using the processor, instances of a content type in the selected keyframes includes detecting face images.

3. The method of claim 1 , further comprising:
selecting, using the processor, instances of a content type from each cluster having a predetermined threshold quantity of similar instances; and
displaying, using the processor, the instances of a content type for each of the clusters having the threshold quantity of similar instances in an order corresponding to cluster size,
wherein cluster size corresponds to quantity of similar instances of the content type in the cluster.

4. The method of claim 3 , wherein selecting and displaying the instances of a content type for each of the clusters includes respectively selecting and displaying images of faces appearing in one or more video clips.

5. The method of claim 4 , wherein displaying images of faces appearing in one or more video clips includes displaying a largest face image and displaying a face image most different from a selected face image.

6. The method of claim 3 , further comprising:
receiving, using the processor, a user selection of at least one of the displayed instances;
identifying, using the processor, a second group of visual media based on correspondence of metadata of the visual media with the text query and the selected instances of a content type;
selecting, using the processor, second keyframes from the second group of identified visual media;
detecting, using the processor, second instances of a content type in the selected second keyframes;
grouping, using the processor, similar second instances of the content type into clusters; and
determining, using the processor, matches between the second instances of the content type and the selected instances of a content type.

7. The method of claim 6 , further comprising:
determining, using the processor, a ranking score for visual media having at least one determined match based on cumulative time during which the selected instances of a content type appear; and
displaying, using the processor, a listing of the visual media based on ranking score.

8. The method of claim 7 , further comprising creating, using the processor, an index of the visual media based on ranking score and a quantity of occurrences that the visual media is viewed, the index including a location of the visual media and one or more locations within the visual media at which a particular instance of a content type appears.

9. The method of claim 7 , further comprising:
analyzing, using the processor, metadata of most-viewed visual media and a name associated with each respective most-viewed visual media prior to receiving the text query;
generating, using the processor, a re-ranked result corresponding to the name;
caching, using the processor, the re-ranked result; and
updating, using the processor, the cached re-ranked result responsive to new text queries.

10. A non-transitory computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions comprising instructions that, if executed by one or more processors, cause the one or more processors to:
retrieve a group of visual media based on correspondence of metadata of the visual media with a text query;
select keyframes from the group of retrieved visual media;
apply face clustering to the keyframes from the group of retrieved visual media;
generate query face images based on the face clusters;
re-rank the group of retrieved visual media on a display based on a received input corresponding to a particular one of the query face images;
index locations within the group of visual media based on the face clusters; and
repurpose images directly from the indexed locations within the group of visual media into a new customized visual media product.

11. The non-transitory computer-readable medium of claim 10 , further comprising instructions that, if executed by one or more processors, cause the one or more processors to
indicate portions of a selected one of the group of retrieved visual media corresponding to a selected query face image.

12. A computing system, comprising:
a display;
a non-transitory computer-readable medium having computer-executable instructions stored thereon; and
a processor coupled to the display and the non-transitory computer-readable medium, wherein the processor executes the instructions to:
retrieve a group of visual media based on correspondence of metadata of the visual media with a text query;
select keyframes from the group of retrieved visual media;
apply face clustering to the keyframes from the group of retrieved visual media;
generate query face images based on face clusters;
re-rank the group of retrieved visual media on the display based on a received input corresponding to a particular one of the query face images;
index locations within the group of visual media based on the face clustering; and
repurpose images directly from the indexed locations within the group of visual media into a new customized visual media product.

13. The computing system of claim 12 , wherein the processor executes the instructions to display portions of a particular retrieved visual media based on a received input corresponding to a particular one of the query face images.

Assignees

• Zhang Tong
• Liu ke-yan
• Sun xin-yun
• Hewlett Packard Development Co

Inventors

• Zhang Tong
• Liu ke-yan
• Sun xin-yun

Classifications

• G06K9/00288

  Physics · mapped topic

• G06F17/30796

  Physics · mapped topic

• G06F17/30268 Primary

  Physics · mapped topic

• G06F16/7844 Primary

  using original textual content or text extracted from visual content or transcript of audio data · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title