Facial recognition using social networking information

Patent metadata

Field	Value
Publication number	US-10133915-B2
Application number	US-201615353066-A
Country	US
Kind code	B2
Filing date	Nov 16, 2016
Priority date	Nov 15, 2011
Publication date	Nov 20, 2018
Grant date	Nov 20, 2018

Abstract

In particular embodiments, one or more images associated with a primary user are received. The image(s) may comprise single images, a series of related images, or video frames. In each image, one or more faces are detected and/or tracked. For each face, a set of one or more candidates are selected who may be identified with the face. A candidate score is calculated for each candidate based on a computed measure of affinity of the primary user for a particular candidate, a facial recognition score comparing the candidate to the face, and a geographic proximity of the candidate to the primary user at a time when the one or more images were created. A winning candidate is selected based on the candidate scores.

First claim

What is claimed is:

1. A method comprising:
by one or more computing devices, receiving one or more images associated with a primary user, wherein one or more faces appear in at least one of the one or more images;
by the one or more computing devices, for each of the one or more faces, selecting one or more candidates who may be identified with the face, wherein the one or more candidates are one or more of:
one or more users of a social network, wherein a computed measure of affinity is based on the primary user's interest in each candidate through the social network, and the computed measure of affinity exceeds a first predetermined threshold; and
one or more public figures, wherein the public figures are each users of the social network that do not have a computed measure of affinity of the primary user for each candidate through the social network that exceeds the first predetermined threshold;

by the one or more computing devices, determining a candidate score for each of the candidates, wherein the candidate score for a particular candidate represents a likelihood that the particular candidate corresponds to the face, the candidate score being based at least in part on:
a geographic proximity between the particular candidate and the primary user at a time that the at least one of the one or more images was created;
the computed measure of affinity of the primary user's interest in the particular candidate; and
a facial recognition comparison of the particular candidate to the face; and

by the one or more computing devices, selecting a winning candidate for the face from the one or more candidates, wherein the selecting is based on the respective candidate scores of the one or more candidates.

2. The method of claim 1 , wherein the one or more candidates are limited to a maximum number of candidates.

3. The method of claim 1 , wherein the one or more images are determined to be related based on:
a common location where the one or more images were captured; or
a common time that the one or more images were captured.

4. The method of claim 3 , wherein determining a candidate score for the particular candidate comprises:
determining a set of individual candidate scores for the particular candidate, wherein each individual candidate score is determined for one of the one or more images in which the face appears; and
determining an aggregate candidate score for the particular candidate based upon individual candidate scores for the particular candidate.

5. The method of claim 4 , wherein the aggregate candidate score is determined only after a minimum threshold number of individual candidate scores have been determined.

6. The method of claim 4 , wherein selecting the winning candidate comprises determining the aggregate candidate score for the candidate based upon a subset of the one or more images.

7. The method of claim 4 , further comprising storing each individual candidate score for a candidate associated with a face in a data store.

8. The method of claim 1 , further comprising presenting an identification of a face as the winning candidate in every image of the one or more images where the face appears.

9. The method of claim 1 , further comprising presenting options to select one of a set of highest-scoring candidates to identify a face in every image of the one or more images where the face appears.

10. The method of claim 1 , wherein a substantial portion of a single scene is depicted in each of the one or more images.

11. The method of claim 1 , wherein the one or more images comprise frames from a video clip.

12. The method of claim 11 , further comprising tracking each face as it appears in two or more frames of the video clip.

13. The method of claim 1 , wherein selecting a particular candidate for the subset of candidates comprises determining a confidence level for the particular candidate, and wherein calculating the candidate score for the particular candidate is further based on the confidence level for the particular candidate.

14. The method of claim 1 , wherein calculating the candidate score for the particular candidate is further based on a minimum threshold confidence level for the particular candidate.

15. The method of claim 1 , wherein determining the winning candidate comprises using a hidden Markov model.

16. The method of claim 1 , wherein the computed measure of affinity is based on a weighted set of one or more predictor functions.

17. The method of claim 1 , wherein calculating the candidate score comprises adjusting the facial recognition score for a candidate by the computed measure of affinity for the candidate.

18. A system comprising:
a processor; and
a non-transitory machine-readable medium configured with instructions to be executed by the processor to:
receive one or more images associated with a primary user, wherein one or more faces appear in at least one of the one or more images;
select, for each of the one or more faces, one or more candidates who may be identified with the face, wherein the one or more candidates are one or more of:
one or more users of a social network, wherein a computed measure of affinity is based on the primary user's interest in each candidate through the social network, and the computed measure of affinity exceeds a first predetermined threshold; and
one or more public figures, wherein the public figures are each users of the social network that do not have a computed measure of affinity of the primary user for each candidate through the social network that exceeds the first predetermined threshold;

determine a candidate score for each of the candidates, wherein the candidate score for a particular candidate represents a likelihood that the particular candidate corresponds to the face, the candidate score being based at least in part on:
a geographic proximity between the particular candidate and the primary user at a time that the at least one of the one or more images was created;
the computed measure of affinity of the primary user's interest in the particular candidate; and
a facial recognition comparison of the particular candidate to the face; and

select a winning candidate for the face from the one or more candidates, wherein the selecting is based on the respective candidate scores of the one or more candidates.

19. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive one or more images associated with a primary user, wherein one or more faces appear in at least one of the one or more images;
select, for each of the one or more faces, one or more candidates who may be identified with the face, wherein the one or more candidates are one or more of:
one or more users of a social network, wherein a computed measure of affinity is based on the primary user's interest in each candidate through the social network, and the computed measure of affinity exceeds a first predetermined threshold; and
one or more public figures, wherein the public figures are each users of the social network that do not have a computed measure of affinity of the primary user for each candidate through the social network that exceeds the first predetermined threshold;

determine a candidate score for each of the candidates, wherein the candidate score for a particular candidate represents a likelihood that the particular candidate corresponds to the face, the candidate score being based at least in part on:
a geographic proximity between the particular candidate and the primary user

Assignees

• Facebook Inc

Inventors

• Garcia David Harry
• St Clair Luke
• Yuen Jenny

Classifications

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V10/768 Primary

  using context analysis, e.g. recognition aided by known co-occurring patterns · CPC title

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06Q50/01

  Physics · mapped topic

• G06K9/00228 Primary

  Physics · mapped topic