Face spoofing detection using a physical-cue-guided multi-source multi-channel framework

Patent metadata

Field	Value
Publication number	US-11250282-B2
Application number	US-202017091140-A
Country	US
Kind code	B2
Filing date	Nov 6, 2020
Priority date	Nov 14, 2019
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

A computer-implemented method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework includes receiving a set of data including face recognition data, liveness data and material data associated with at least one face image, obtaining a shared feature from the set of data using a backbone neural network structure, performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction, and aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance.

First claim

What is claimed is:

1. A computer-implemented method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework, comprising:
receiving a set of data including face recognition data, liveness data and material data associated with at least one face image;
obtaining a shared feature from the set of data using a backbone neural network structure;
performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction; and
aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance,
wherein the first proxy task generate a depth prediction together with a depth channel attention matrix, the liveness prediction task generates a liveness feature and a corresponding liveness channel attention matrix, and the material prediction task generates a liveness feature and a corresponding material channel attention matrix.

2. The method as recited in claim 1 , wherein the liveness data includes depth ground truth and material ground truth prepared off-line, and the material data includes a Materials in Context (MINC) dataset.

3. The method as recited in claim 1 , wherein obtaining the shared feature further includes extracting the shared feature using a shared feature extractor.

4. The method as recited in claim 1 , wherein aggregating the outputs includes combining the depth channel attention matrix, the liveness channel attention matrix and the material channel attention matrix using a prediction-level attention model.

5. The method as recited in claim 1 , wherein the face spoofing detection implements an overall loss design including a face recognition loss, an l 1 -based reconstruction loss, at least one cross-entropy loss and at least one multi-class softmax loss.

6. The method as recited in claim 1 , wherein the face spoofing detection is implemented within a biometric system.

7. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework, the method performed by the computer comprising:
receiving a set of data including face recognition data, liveness data and material data associated with at least one face image;
obtaining a shared feature from the set of data using a backbone neural network structure;
performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction; and
aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance,
wherein the first proxy task generate a depth prediction together with a depth channel attention matrix, the liveness prediction task generates a liveness feature and a corresponding liveness channel attention matrix, and the material prediction task generates a liveness feature and a corresponding material channel attention matrix.

8. The computer program product as recited in claim 7 , wherein the liveness data includes depth ground truth and material ground truth prepared off-line, and the material data includes a Materials in Context (MINC) dataset.

9. The computer program product as recited in claim 7 , wherein obtaining the shared feature further includes extracting the shared feature using a shared feature extractor.

10. The computer program product as recited in claim 7 , wherein aggregating the outputs includes combining the depth channel attention matrix, the liveness channel attention matrix and the material channel attention matrix using a prediction-level attention model.

11. The computer program product as recited in claim 7 , wherein the face spoofing detection implements an overall loss design including a face recognition loss, an l 1 -based reconstruction loss, at least one cross-entropy loss and at least one multi-class softmax loss.

12. The computer program product as recited in claim 7 , wherein the face spoofing detection is implemented within a biometric system.

13. A system for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework, comprising:
a memory device storing program code; and
at least one processor device operatively coupled to the memory device and configured to execute program code stored on the memory device to:
receive a set of data including face recognition data, liveness data and material data associated with at least one face image;
obtain a shared feature from the set of data using a backbone neural network structure;
perform, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction; and
aggregate outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance,
wherein the first proxy task generate a depth prediction together with a depth channel attention matrix, the liveness prediction task generates a liveness feature and a corresponding liveness channel attention matrix, and the material prediction task generates a liveness feature and a corresponding material channel attention matrix, and wherein aggregating the outputs includes combining the depth channel attention matrix, the liveness channel attention matrix and the material channel attention matrix using a prediction-level attention model.

14. The system as recited in claim 13 , wherein the liveness data includes depth ground truth and material ground truth prepared off-line, and the material data includes a Materials in Context (MINC) dataset.

15. The system as recited in claim 13 , wherein obtaining the shared feature further includes extracting the shared feature using a shared feature extractor.

16. The system as recited in claim 13 , wherein the face spoofing detection implements an overall loss design including a face recognition loss, an l 1 -based reconstruction loss, at least one cross-entropy loss and at least one multi-class softmax loss.

17. The system as recited in claim 13 , wherein the face spoofing detection is implemented within a biometric system.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Yu Xiang
• Liu Buyu
• Chandraker Manmohan
• Wu Junru

Classifications

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V40/45

  Detection of the body part being alive · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V40/169

  Holistic features and representations, i.e. based on the facial image taken as a whole · CPC title