Ranking clusters based on facial image analysis

Patent metadata

Field	Value
Publication number	US-9465993-B2
Application number	US-78449810-A
Country	US
Kind code	B2
Filing date	May 21, 2010
Priority date	Mar 1, 2010
Publication date	Oct 11, 2016
Grant date	Oct 11, 2016

Abstract

A user's collection of images may be analyzed to identify people's faces within the images, then create clusters of similar faces, where each of the clusters may represent a person. The clusters may be ranked in order of size to determine a relative importance of the associated person to the user. The ranking may be used in many social networking applications to filter and present content that may be of interest to the user. In one use scenario, the clusters may be used to identify images from a second user's image collection, where the identified images may be pertinent or interesting to the first user. The ranking may also be a function of user interactions with the images, as well as other input not related to the images. The ranking may be incrementally updated when new images are added to the user's collection.

First claim

What is claimed is:

1. A method comprising:
by the at least one computer processing device:
receiving image metadata from an image collection that is associated with a user, the image metadata comprising processed face objects from images of the image collection;
analyzing the image metadata to identify similar face objects, the similar face objects having one or more matching criteria with respect to a threshold;
grouping the similar face objects into clusters, wherein, within the image collection that is associated with the user, individual clusters have different numbers of occurrences of the similar face objects;
determining person identities associated with the individual clusters; and
ranking the person identities based on the different numbers of occurrences of the similar face objects within the image collection that is associated with the user, the person identities including a first person identity of a first person other than the user and a second person identity of a second person other than the user, wherein the first person identity is associated with a first individual cluster having a first number of occurrences of face objects of the first person and the second person identity is associated with a second individual cluster having a second number of occurrences of face objects of the second person, wherein the first number of occurrences is greater than the second number of occurrences and the first person identity is ranked higher than the second person identity.

2. The method of claim 1 , further comprising:
by the at least one computer processing device:
prioritizing, for the user, first messages relating to the first person over second messages relating to the second person based on the ranking.

3. The method of claim 2 , wherein the image collection is associated with a social network account of the user for a social networking application and the prioritizing comprises prioritizing the first messages over the second messages within the social networking application.

4. The method of claim 3 , wherein the prioritizing comprises prioritizing the first messages over the second messages within a newsfeed of the social networking application.

5. The method of claim 1 , further comprising:
by the at least one computer processing device:
emphasizing first items related to the first person identity; and
not emphasizing second items related to the second person identity.

6. The method of claim 1 , wherein:
the user has an associated social networking account that includes the image collection, and
the person identities are ranked according to at least the different numbers of occurrences of the similar face objects within the image collection of the social networking account.

7. A system comprising:
at least one hardware processor; and
at least one memory or non-volatile storage media storing computer-readable instructions which, when executed by the at least one hardware processor, cause the at least one hardware processor to:
obtain image metadata for images of an image collection that is associated with a user, the image metadata comprising processed face objects from the images of the image collection;
analyze the image metadata to identify similar face objects, using one or more matching criteria;
group the similar face objects into clusters, wherein, within the image collection that is associated with the user, individual clusters having different sizes based on numbers of occurrences of the similar face objects that are associated with different people; and
rank the different people based on the different sizes of the individual clusters, the different people including a first person other than the user and a second person other than the user, wherein the first person is associated with a first individual cluster having a first number of occurrences of face objects of the first person and the second person is associated with a second individual cluster having a second number of occurrences of face objects of the second person, wherein the first number of occurrences is greater than the second number of occurrences and the first person is ranked higher than the second person.

8. The system of claim 7 , wherein the computer-readable instructions further cause the at least one hardware processor to:
produce the image metadata by:
analyzing individual images to identify faces within the individual images,
processing the faces to determine face vectors for the faces, and
storing the face vectors in the image metadata.

9. The system of claim 7 , wherein the image collection is associated with a social network account of the user for a social networking application.

10. The system of claim 9 , wherein the computer-readable instructions further cause the at least one hardware processor to:
prioritize, for the user, first status updates in the social networking application relating to the first person over second status updates in the social networking application relating to the second person.

11. The system of claim 10 , wherein the computer-readable instructions further cause the at least one hardware processor to:
prioritize the first status updates over the second status updates in a social networking newsfeed of the social networking application.

12. The system of claim 7 , wherein the computer-readable instructions further cause the at least one hardware processor to:
analyze other images of another image collection of another user; and
rank first other images of the another image collection that include the first person relatively higher than second other images of the another image collection that include the second person.

13. The system of claim 7 , wherein the computer-readable instructions further cause the at least one hardware processor to:
analyze other images of another image collection of another user to determine a confidence threshold; and
use the confidence threshold to identify the similar face objects.

14. The system of claim 7 , wherein the computer-readable instructions further cause the at least one hardware processor to:
incrementally update the ranking as the different sizes of the individual clusters change responsive to new images being added to the image collection.

15. One or more computer-readable memory devices or storage devices having instructions stored thereon that, when executed by a computing device, cause the computing device to perform acts comprising:
obtaining image metadata for images of an image collection that is associated with a user, the image metadata comprising processed face objects from the images of the image collection;
analyzing the image metadata to identify different clusters of images in the image collection that is associated with the user, the different clusters representing different people and having different associated cluster sizes in the image collection of the user, wherein the different clusters of images including at least a first cluster of images representing a first person other than the user and a second cluster of images representing a second person other than the user, wherein the first cluster has a first cluster size and the second cluster has a second cluster size; and
ranking the different people based on the different associated cluster sizes,
wherein the different associated cluster sizes are based on corresponding numbers of occurrences of corresponding face objects of the different people in the image collection that is associated with the user, the first cluster having a first number of corresponding occurrences of first face objects of the first person other than the use

Assignees

• Krupka Eyal
• Abramovski Igor
• Kviatkovsky Igor
• Microsoft Technology Licensing Llc

Inventors

• Krupka Eyal
• Abramovski Igor
• Kviatkovsky Igor

Classifications

• G06F18/23

  Clustering techniques · CPC title

• G06V20/30 Primary

  in albums, collections or shared content, e.g. social network photos or video · CPC title

• G06F18/2113

  by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06K9/00677 Primary

  Physics · mapped topic