Method and a system for face verification

Patent metadata

Field	Value
Publication number	US-10289897-B2
Application number	US-201615366944-A
Country	US
Kind code	B2
Filing date	Dec 1, 2016
Priority date	Jun 16, 2014
Publication date	May 14, 2019
Grant date	May 14, 2019

Abstract

Disclosed is an apparatus for face verification. The apparatus may comprise a feature extraction unit and a verification unit. In one embodiment, the feature extraction unit comprises a plurality of convolutional feature extraction systems trained with different face training set, wherein each of systems comprises: a plurality of cascaded convolutional, pooling, locally-connected, and fully-connected feature extraction units configured to extract facial features for face verification from face regions of face images; wherein an output unit of the unit cascade, which could be a fully-connected unit in one embodiment of the present application, is connected to at least one of previous convolutional, pooling, locally-connected, or fully-connected units, and is configured to extract facial features (referred to as deep identification-verification features or DeepID2) for face verification from the facial features in the connected units. The verification unit may be configured to compare the obtained DeepID2 extracted from two face images to be compared to determine if the two face images are from the same identity or not.

First claim

What is claimed is:

1. An apparatus for face verification, comprising:
at least one processor and a memory having processor-executable instructions stored therein, and the instructions when executed by the at least one processor, configure the apparatus to:
extract DeepID2 from different regions of face images by using differently trained convolutional feature extraction systems, wherein each of the systems comprises:
a layer cascade comprising a plurality of convolutional layers, a plurality of pooling layers, a plurality of locally-connected layers and a plurality of fully-connected layers, wherein an output layer of the layer cascade is connected to at least one of a previous convolutional, the pooling, the locally-connected, or the fully connected layers, and is configured to extract facial features as the DeepID2 for face verification from the facial features in the connected layers, wherein the fully-connected layers are directly connected to the locally-connected layers and the pooling layers, and are configured to receive an output of the locally-connected layers as a first input and receive an output of the pooling layers as a second input; and

compare the facial features extracted by said output layer from two face images to be compared to determine if the two face images are from a same identity or not.

2. The apparatus of claim 1 , wherein the output layer comprises a fully-connected layer.

3. The apparatus of claim 1 , wherein the apparatus is further configured to: input pairs of face regions, an identification supervisory signal and a verification supervisory signal to the convolutional feature extraction systems to adjust weights on connections between neurons of the convolutional feature extraction systems.

4. A method for face verification, executed by a face verification processor, comprising:
extracting DeepID2 from different regions of face images by using differently trained convolutional feature extraction systems, wherein each of systems comprises a layer cascade comprising a plurality of convolutional layers, a plurality of pooling layers, a plurality of locally-connected layers, and a plurality of fully-connected layers, wherein an output layer of the layer cascade is connected to at least one of a previous convolutional, the pooling, the locally-connected, or the fully-connected layers, and is configured to extract facial features as the DeepID2 for face verification from the facial features in the connected layers, wherein the fully-connected layers are directly connected to the plurality of locally-connected layers and the plurality of pooling layers, and are configured to receive an output of the locally-connected layers as a first input and receive an output of the plurality of pooling layers as a second input and
comparing DeepID2 extracted from two face images to be compared, respectively, to determine if the two face images are from the same identity or not.

5. The apparatus of claim 1 , wherein each of the convolutional layers is connected to a pooling layer, and an output of each convolution layer is inputted into the pooling layer.

6. The apparatus of claim 3 , wherein the output layer is followed by an n-way softmax layer for classifying the DeepID2 extracted from each face region into one out of all classes of face identities; and wherein the apparatus is further configured to compare the classified identity and a given ground-truth identity to generate identification errors, the generated identification errors being back-propagated through the convolutional feature extraction system to adjust weights on connections between the neurons of the convolutional feature extraction system.

7. The apparatus of claim 3 , wherein the apparatus is further configured to generate verification errors by comparing dissimilarities between two DeepID2 vectors extracted from two face regions, respectively; and
wherein the generated verification errors are back-propagated through the convolutional feature extraction system to adjust weights on connections between the neurons of the convolutional feature extraction system.

8. The apparatus of claim 3 , wherein for each of the convolutional feature extraction systems, the processor and the each of said systems co-operate to:
1) sample two face region-label pairs from a predetermined training set;
2) extract the DeepID2 from the two face regions in the two sampled face region-label pairs, respectively;
3) generate identification errors and verification errors based on the DeepID2 extracted from the two face regions;
4) back-propagate the identification errors and the verification errors through the convolutional feature extraction system to adjust weights on connections between the neurons of the convolutional feature extraction system; and
5) repeat steps 1)-4) until the training is converged such that the weights on connections between the neurons of the convolutional feature extraction system are determined.

9. The apparatus of claim 1 , wherein the apparatus is further configured to:
select one or more groups of DeepID2 from the extracted DeepID2, each group containing the DeepID2 extracted from a plurality of face regions of each face image;
compare the selected one or more groups of DeepID2 to output one or more face verification scores; and
fuse the one or more face verification scores to make a single face verification decision.

10. The apparatus of claim 1 , wherein each of the convolutional layers contains a plurality of neurons with local receptive fields and shared connection weights among the neurons or subsets of the neurons in the convolutional layer.

11. The apparatus of claim 7 , wherein the dissimilarities between the two DeepID2 vectors comprise negative of L1 norm, L2 norm, or cosine similarity between the two DeepID2 vectors.

12. The method of claim 4 , wherein the method further comprises:
training a plurality of convolutional feature extraction systems for simultaneous identity classification and verification by inputting pairs of face regions, an identification supervisory signal and a verification supervisory signal so as to adjust weights on connections between neurons of the convolutional feature extraction systems.

13. The method of claim 12 , wherein the output layer comprises a fully-connected layer.

14. The method of claim 12 , wherein the training further comprises:
classifying the DeepID2 extracted from each face region into one out of all classes of face identities;
comparing the classified identity and a given ground-truth identity to generate identification errors; and
back-propagating the generated identification errors through the convolutional feature extraction system to adjust weights on connections between the neurons of the convolutional feature extraction system.

15. The method of claim 12 , wherein the training further comprises:
comparing dissimilarities between two DeepID2 vectors extracted from two face regions to be compared, respectively, to generate verification errors; and
back-propagating the generated verification errors through each convolutional feature extraction system to adjust weights on connections between the neurons of the convolutional feature extraction system.

16. The method of claim 12 , wherein the training further comprises:
classifying the DeepID2 extracted from each face region into one out of all classes of face identities;
comparing the classified identity and a given ground-truth identity to generate identification errors;
comparing dissimilarities between two DeepID2 vectors extracted from two face regions to be compared, respectiv

Assignees

• Beijing Sensetime Tech Development Co Ltd

Inventors

• Tang Xiaoou
• Sun Yi
• Wang Xiaogang

Classifications

• G06V40/171 Primary

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06V10/20

  Image preprocessing · CPC title