Face clustering with image uncertainty

Patent metadata

Field	Value
Publication number	US-11810398-B2
Application number	US-202117526492-A
Country	US
Kind code	B2
Filing date	Nov 15, 2021
Priority date	Nov 16, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Methods and systems for face clustering include determining a quality score for each of a set of input images. A first subset of the input images is clustered, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters. A second subset of the input images is clustered, having respective quality scores below the predetermined threshold. An action is performed responsive to the clustered images after the second subset is added to the initial set of clusters.

First claim

What is claimed is:

1. A computer-implemented method for face clustering, comprising:
determining a quality score for each of a set of input images, including determining a probabilistic face embedding for each of the set of input images and normalizing an uncertainty of the probabilistic face embedding for each of the set of input images;
clustering a first subset of the input images, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters;
clustering a second subset of the input images, having respective quality scores below the predetermined threshold; and
performing an action responsive to the clustered images after the second subset is added to the initial set of clusters.

2. The method of claim 1 , wherein determining the probabilistic face embedding for each of the set of input images includes determining a mean and a standard deviation.

3. The method of claim 2 , wherein determining the quality score includes dividing a difference between a raw quality score and a mean value by a standard deviation for the probabilistic face embedding for each of the set of input images.

4. The method of claim 1 , wherein clustering the second subset of the input images includes adding the second subset of the input images to a same new cluster, distinct from the initial set of clusters.

5. The method of claim 1 , wherein clustering the second subset of the input images includes adding images of the second subset to respective clusters of the initial set of clusters.

6. The method of claim 1 , further comprising comparing each image of the second subset to a reference image of each of the initial set of clusters to determine a respective cluster for assignment.

7. The method of claim 1 , wherein clustering the first subset of the images includes determining similarities between images in the first subset of the input images and applying different similarity thresholds for pairs of images in the first subset in different respective uncertainty regimes.

8. The method of claim 1 , wherein the action is selected from the group consisting of a security action, a promotional action, a health & safety action, and a crowd control action.

9. A computer-implemented method for face clustering, comprising:
determining a quality score for each of a set of input images, including determining a probabilistic face embedding for each of the set of input images;
clustering a first subset of the input images, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters, including determining similarities between images in the first subset of the input images and applying different similarity thresholds for pairs of images in the first subset in different respective uncertainty regimes;
clustering a second subset of the input images, having respective quality scores below the predetermined threshold, by adding images of the second subset to respective clusters of the initial set of clusters;
performing face recognition on a new image by comparing the new image to references images of each of the initial set of clusters; and
performing an action responsive to the clustered images after the second subset is added to the initial set of clusters.

10. A system for face clustering, comprising:
a hardware processor; and
a memory that stores a computer program, which, when executed by the hardware processor, causes the hardware processor to:
determine a quality score for each of a set of input images, including a determination of a probabilistic face embedding for each of the set of input input and a normalization of an uncertainty of the probabilistic face embedding for each of the set of input images;
cluster a first subset of the input images, having respective quality scores that exceed a predetermined threshold, to form an initial set of clusters;
cluster a second subset of the input images, having respective quality scores below the predetermined threshold; and
perform an action responsive to the clustered images after the second subset is added to the initial set of clusters.

11. The system of claim 10 , wherein the computer program further causes the hardware processor to determine a mean and a standard deviation of the probabilistic face embedding.

12. The system of claim 11 , wherein the computer program further causes the hardware processor to divide a difference between a raw quality score and a mean value by a standard deviation for the probabilistic face embedding for each of the set of input images.

13. The system of claim 10 , wherein the computer program further causes the hardware processor to add the second subset of the input images to a same new cluster, distinct from the initial set of clusters.

14. The system of claim 10 , wherein the computer program further causes the hardware processor to add images of the second subset to respective clusters of the initial set of clusters.

15. The system of claim 10 , wherein the computer program further causes the hardware processor to compare each image of the second subset to a reference image of each of the initial set of clusters to determine a respective cluster for assignment.

16. The system of claim 10 , wherein the computer program further causes the hardware processor to perform face recognition on an input image by comparing the input image to reference images of each of the initial set of clusters.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Debnath Biplob
• Chakradhar Srimat
• Coviello Giuseppe
• Yang Yi

Classifications

• G06V40/172 Primary

  Classification, e.g. identification · CPC title

• G06F18/10

  Pre-processing; Data cleansing · CPC title

• G06F18/2321

  using statistics or function optimisation, e.g. modelling of probability density functions · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06V10/762 Primary

  using clustering, e.g. of similar faces in social networks · CPC title