Method for recognizing face from monitoring video data

Patent metadata

Field	Value
Publication number	US-11263435-B2
Application number	US-202016941581-A
Country	US
Kind code	B2
Filing date	Jul 29, 2020
Priority date	Jul 31, 2019
Publication date	Mar 1, 2022
Grant date	Mar 1, 2022

Abstract

A method for recognizing a face from monitoring video data is disclosed. Two neural networks are used to compare and score high-dimensional face features of a face, and a K-neighbor algorithm and a screening mechanism with a Euclidean distance as a threshold are combined for face comparison recognition to obtain an accurate face recognition result. In addition, the present disclosure also performs further screening based on the time of video data and the frequency of face appearance, and finally obtains a recognition result, thereby obtaining a more accurate face recognition result. The present disclosure can perform relatively accurate face recognition on video data or picture data captured by a real-time or historical monitoring camera.

First claim

What is claimed is:

1. A method for recognizing a face from monitoring video data, comprising:
step 1, collecting in a monitoring area, a self-made face photo set, obtaining a public face data set, pre-processing the self-made face photo set and the public face data set, using a face detection algorithm and a feature extractor to extract high-dimensional features of a face from a pre-processed face data set, and saving as a training set of a model, wherein the high-dimensional features are 512 dimensions features;
step 2, collecting face-containing video data using a monitoring camera device in the monitoring area, extracting face pictures in a video using an existing face detection algorithm, filtering out through simple manual screening, pictures in which a face is hard to be seen, manually marking tags of the face pictures, and saving as a verification set of the model finally;
step 3, selecting two different face comparison models, namely a first face comparison model and a second face comparison model;
step 4, training the two face comparison models in step 3 using the training set in step 1 respectively, verifying the two face comparison models in step 3 using the verification set in step 2, and then saving the obtained two face comparison models;
step 5, collecting K standard face photos of each trusted identity, extracting high-dimensional features of a standard face from each standard face photo using a feature extractor, forming a face matrix by standard face photos, the high-dimensional features of the standard faces and corresponding identity tags, saving the face matrix, and thus establishing a face standard database;
step 6, collecting video data in real time using the monitoring camera device in the monitoring area, acquiring each frame of a picture in a video stream frame by frame, and extracting high-dimensional features of all faces to be recognized in each frame of the picture using the face detection algorithm and the feature extractor;
step 7, executing following operations respectively on the high-dimensional features of each face to be recognized obtained in step 6;
step 7.1, jointly inputting the high-dimensional features of a face to be recognized and the face matrix obtained in step 5 into the two face comparison models saved in step 4, outputting by each face comparison model, a matched score value between the high-dimensional features of the face to be recognized and the high-dimensional features of each standard face of the face standard database;
step 7.2, performing weighted fusion of the matched score value of the first face comparison model obtained in step 7.1 and the matched score value of the second face comparison model to obtain a comprehensive matched score value between the high-dimensional features of the face to be recognized and the high-dimensional features of each standard face of the face standard set;
step 7.3, selecting the identity tags corresponding to the high-dimensional features of the L standard faces with high comprehensive matched score values as preliminary recognition identity tags for model recognition;
step 7.4, jointly inputting the high-dimensional features of a face to be recognized and the face matrix obtained in step 5 into the K-neighbor algorithm, wherein in the K-neighbor algorithm, a Euclidean distance E1 between the high-dimensional features of the face to be recognized and the high-dimensional features of each standard face of the face standard database is first calculated, the identity tags corresponding to the high-dimensional features of K standard faces with a small Euclidean distance E1 are then selected, if K−1 identity tags among the K identity tags are the same, the K-neighbor algorithm outputs the same identity tag as the preliminary recognition identity tag for algorithm recognition, and a Euclidean distance average value E of the K−1 identity tags is calculated, otherwise, the K-neighbor algorithm outputs no solution;
step 7.5, when the preliminary recognition identity tag for algorithm recognition obtained in step 7.4 exists in the preliminary recognition identity tag for model recognition obtained in step 7.3, comparing the Euclidean distance average value E corresponding to the preliminary recognition identity tag for algorithm recognition with a set similarity threshold:
when the Euclidean distance average value E is greater than the set similarity threshold, using the preliminary recognition identity tag for algorithm recognition as a recognition result of the high-dimensional features of the face to be recognized, forming one record by the high-dimensional features of the face to be recognized, the comprehensive matched score value, the Euclidean distance average value and the identity tag, and putting the record into a recognizable set;
otherwise, providing no solution for the high-dimensional features of the face to be recognized, forming one record by the high-dimensional features of the face to be recognized, and putting the record into an unrecognized set,
K and L being set positive integers more than 1.

2. The method as claimed in claim 1 , wherein two different face comparison models established in step 3 are as follows:
a calculation process of the first face comparison model is as follows: first calculating a product, sum, absolute difference and square of the difference between the high-dimensional features of the faces of two people and splicing into matrix data, then performing an activation function which is a convolution calculation and batch normalization of a modified linear unit, then performing an activation function which is a convolution calculation and batch normalization calculation of a linear regression, and finally outputting a score value by a fully connected layer in which an activation function is an S-type function; and
a calculation process of the second face comparison model is as follows: first calculating a product, sum, absolute difference and square of the difference between the high-dimensional features of the faces of two people and splicing into matrix data, then performing an activation function which is a convolution calculation and batch normalization of a modified linear unit, then performing squeeze and excitation module calculation of an attention mechanism, then performing an activation function which is a convolution calculation and batch normalization calculation of a linear regression, and finally outputting a score value by a fully connected layer in which an activation function is an S-type function.

3. The method as claimed in claim 1 , wherein a value of K is 4, and a value of L is 3.

4. The method as claimed in claim 1 , further comprising a step of judging whether a recognition result in the recognizable set is output:
step 8, taking out at an interval of 1 s, all the records of previous m seconds from the recognizable set obtained in step 7, performing classification according to the identity tags, and saving high-dimensional features of a face to be recognized with a highest comprehensive matched score value of each identity tag as a preliminary screening result of each second;
step 9, respectively calculating a Euclidean distance E2 between two of the high-dimensional features of the face to be recognized in the preliminary screening result of the each second obtained in step 8:
if the Euclidean distance E2 is less than or equal to the set similarity threshold, indicating that the high-dimensional features of two faces to be recognized are the same person, and retaining the identity tags with relatively high comprehensive matched score values as valid recognition results of the high-dimensional features of the face to be recognized of the each second;
if the Euclidean distance E2 is greater than the set similarity threshold, indicating that the high-dimensional features of two faces to be

Assignees

• Univ Guangxi

Inventors

• Fan Zuojun
• Gao Jian
• Jiang Wenjun
• Liang Guanglin

Classifications

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06V10/763

  Non-hierarchical techniques, e.g. based on statistics of modelling distributions · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V40/166 Primary

  using acquisition arrangements · CPC title

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title