System and method for identifying faces in unconstrained media

Patent metadata

Field	Value
Publication number	US-9449432-B2
Application number	US-201414576818-A
Country	US
Kind code	B2
Filing date	Dec 19, 2014
Priority date	Dec 19, 2013
Publication date	Sep 20, 2016
Grant date	Sep 20, 2016

Abstract

Methods and systems for facial recognition are provided. The method includes determining a three-dimensional (3D) model of a face of an individual based on different images of the individual. The method also includes extracting two-dimensional (2D) patches from the 3D model. Further, the method includes generating a plurality of signatures of the face using different combinations of the 2D patches, wherein the plurality of signatures correspond to respective views of the 3D model from different angles.

First claim

What is claimed is:

1. A method comprising:
determining a three-dimensional (3D) model of a face of an individual based on a plurality of different images of the individual;
extracting two-dimensional (2D) patches from the 3D model; and
generating a plurality of signatures of the face using different combinations of the 2D patches, wherein the plurality of signatures correspond to respective views of the 3D model from different angles;
determining a plurality of attributes of the individual that semantically describe characteristics of the individual; and
indexing the plurality of signatures based on the plurality of attributes.

2. The method of claim 1 , wherein the determining the 3D model comprises:
identifying elements of the 3D model lacking information from the plurality of images; and
providing the information for the identified elements using domain knowledge compiled from individuals having attributes similar to those of the individual.

3. The method of claim 1 , further comprising modifying the 3D model by normalizing lighting variations in the 3D model.

4. The method of claim 1 , further comprising neutralizing a facial expression resulting from the plurality of different images of the individual.

5. The method of claim 1 , further comprising modifying the 3D model based on an age of the individual.

6. The method of claim 1 , further comprising determining respective uncertainty values for the plurality of signatures, wherein the uncertainty values are based on a quality of respective 2D patches included in the plurality of signatures.

7. The method of claim 1 , further comprising determining that a face image matches at least one of the plurality of signatures.

8. The method of claim 7 , wherein the determining that the face image matches comprises modifying a resolution of the plurality of signatures based on a resolution of the face image.

9. The method of claim 7 , wherein the determining that the face image matches comprises matching using a plurality imaging modalities.

10. The method of claim 1 , wherein the plurality of signatures of the face are iteratively refined using a number of additional face images of the individual.

11. The method of claim 10 , wherein the plurality of signatures of the face has a fixed size irrespective of the number of additional face images.

12. The method of claim 1 , further comprising:
determining uncertainty metrics corresponding, respectively, to the plurality of signatures;
associating the plurality of signatures with the corresponding uncertainty metrics.

13. The method of claim 1 , further comprising determining which of the plurality of signatures corresponds to a portion of the face having a greatest number of discriminative features.

14. A facial recognition system comprising:
a processor;
a storage system;
program instructions stored on a computer readable hardware storage device for execution by the processor, the program instructions comprising:
program instructions that determine a three-dimensional (3D) model of a face of an individual based on a plurality of different images of the individual;
program instructions that extract two-dimensional (2D) patches from the 3D model; and
program instructions that generate a plurality of signatures of the face using different combinations of the 2D patches, wherein the plurality of signatures correspond to respective views of the 3D model from different angles;

determining a plurality of attributes of the individual that semantically describe characteristics of the individual; and
indexing the plurality of signatures based on the plurality of attributes.

15. The system of claim 14 , wherein the determining the 3D model comprises:
identifying elements of the 3D model lacking information from the plurality of images; and
providing the information for the identified elements using domain knowledge compiled from individuals having attributes similar to those of the individual.

16. The system of claim 14 , further comprising modifying the 3D model by normalizing lighting variations in the 3D model.

17. The system of claim 14 , further comprising normalizing a facial expression resulting from the plurality of different images of the individual.

18. The system of claim 14 , further comprising modifying the 3D model based on an age of the individual.

19. The system of claim 14 , further comprising determining respective uncertainty values for the plurality of signatures, wherein the uncertainty values are based on a quality of respective 2D patches included in the plurality of signatures.

20. The system of claim 14 , further comprising determining that a face image matches at least one of the plurality of signatures.

21. The system of claim 20 , wherein the determining that the face image matches comprises modifying a resolution of the plurality of signatures based on a resolution of the face image.

22. The system of claim 20 , wherein the determining that the face image matches comprises matching using a plurality imaging modalities.

23. The system of claim 14 , wherein the plurality of signatures of the face are iteratively refined using a number of additional face images of the individual.

24. The system of claim 14 , wherein the plurality of signatures of the face has a fixed size irrespective of the number of additional face images.

25. The system of claim 14 , further comprising:
determining uncertainty metrics corresponding, respectively, to the plurality of signatures;
associating the plurality of signatures with the corresponding uncertainty metrics.

26. The system of claim 14 , further comprising determining which of the plurality of signatures corresponds to a portion of the face having a greatest number of discriminative features.

Assignees

• Avigilon Fortress Corp

Inventors

• Kanaujia Atul
• Ramanathan Narayanan
• Choe Tae Eun

Classifications

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06T2200/04

  involving 3D image data · CPC title

• G06T2207/30201

  Face · CPC title