Method and system for facial recognition

Patent metadata

Field	Value
Publication number	US-10776609-B2
Application number	US-201815905609-A
Country	US
Kind code	B2
Filing date	Feb 26, 2018
Priority date	Feb 26, 2018
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

One embodiment provides a method for face liveness detection. The method comprises receiving a first image comprising a face of a user, determining one or more two-dimensional (2D) facial landmark points based on the first image, and determining a three-dimensional (3D) pose of the face in the first image based on the one or more determined 2D facial landmark points and one or more corresponding 3D facial landmark points in a 3D face model for the user. The method further comprises determining a homography mapping between the one or more determined 2D facial landmark points and one or more corresponding 3D facial landmark points that are perspectively projected based on the 3D pose, and determining liveness of the face in the first image based on the homography mapping.

First claim

What is claimed is:

1. A method for face liveness detection, comprising:
receiving a first image comprising a face of a user;
determining one or more two-dimensional (2D) facial landmark points based on the first image;
determining a three-dimensional (3D) pose of the face in the first image based on the one or more 2D facial landmark points and a first set of 3D facial landmark points, wherein the first set comprises one or more 3D facial landmark points in a 3D face model of the face of the user, and the one or more 3D facial landmark points of the first set correspond to the one or more 2D facial landmark points;
determining a homography mapping between the one or more 2D facial landmark points and a second set of 3D facial landmark points, wherein the second set comprises one or more other 3D facial landmark points that are perspectively projected from the one or more 3D facial landmark points of the first set based on the 3D pose; and
determining liveness of the face in the first image based on the homography mapping to determine actual presence of the user during capture of the first image.

2. The method of claim 1 , further comprising:
transforming the one or more 2D facial landmark points to one or more other 2D facial landmark points based on one or more parameters of an image sensor that captured the first image to generate a second image including the one or more other 2D facial landmark points;
wherein the homography mapping is further based on the second image.

3. The method of claim 1 , further comprising:
receiving a request to register a new user, wherein the request comprises multiple facial images of the new user at different poses;
determining and tracking at least one 2D facial landmark point in the multiple facial images; and
generating a 3D face model of a face of the new user based on the at least one 2D facial landmark point and one or more parameters of a sensor.

4. The method of claim 1 , further comprising:
refining the 3D pose and the homography mapping to reduce a distance between the one or more 2D facial landmark points and the one or more other 3D facial landmark points of the second set.

5. The method of claim 1 , wherein determining liveness of the face in the first image comprises:
determining a deviation of the homography mapping from an identity matrix to detect whether the face in the first image is one of a 3D face presented to an image sensor that captured the first image or a 2D facial image of the user presented to the image sensor.

6. The method of claim 1 , wherein determining a homography mapping comprises decomposing the homography mapping based on one or more parameters of an image sensor that captured the first image.

7. The method of claim 1 , further comprising:
performing facial verification based on the first image to determine the user is a registered user; and
retrieving the 3D face model of the face of the user in response to determining the user is a registered user.

8. A system for face liveness detection, comprising:
at least one processor; and
a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including:
receiving a first image comprising a face of a user;
determining one or more two-dimensional (2D) facial landmark points based on the first image;
determining a three-dimensional (3D) pose of the face in the first image based on the one or more 2D facial landmark points and a first set of 3D facial landmark points, wherein the first set comprises one or more 3D facial landmark points in a 3D face model of the face of the user, and the one or more 3D facial landmark points of the first set correspond to the one or more 2D facial landmark points;
determining a homography mapping between the one or more 2D facial landmark points and a second set of 3D facial landmark points, wherein the second set comprises one or more other 3D facial landmark points that are perspectively projected from the one or more 3D facial landmark points of the first set based on the 3D pose; and
determining liveness of the face in the first image based on the homography mapping to determine actual presence of the user during capture of the first image.

9. The system of claim 8 , wherein the operations further include:
transforming the one or more 2D facial landmark points to one or more other 2D facial landmark points based on one or more parameters of an image sensor that captured the first image to generate a second image including the one or more other 2D facial landmark points;
wherein the homography mapping is further based on the second image.

10. The system of claim 8 , wherein the operations further include:
receiving a request to register a new user, wherein the request comprises multiple facial images of the new user at different poses;
determining and tracking at least one 2D facial landmark point in the multiple facial images; and
generating a 3D face model of a face of the new user based on the at least one 2D facial landmark point and one or more parameters of a sensor.

11. The system of claim 8 , wherein the operations further include:
refining the 3D pose and the homography mapping to reduce a distance between the one or more 2D facial landmark points and the one or more other 3D facial landmark points of the second set.

12. The system of claim 8 , wherein determining liveness of the face in the first image comprises:
determining a deviation of the homography mapping from an identity matrix to detect whether the face in the first image is one of a 3D face presented to an image sensor that captured the first image or a 2D facial image of the user presented to the image sensor.

13. The system of claim 8 , wherein determining a homography mapping comprises decomposing the homography mapping based on one or more parameters of an image sensor that captured the first image.

14. The system of claim 8 , wherein the operations further include:
performing facial verification based on the first image to determine the user is a registered user; and
retrieving the 3D face model of the face of the user in response to determining the user is a registered user.

15. A non-transitory computer readable storage medium including instructions to perform a method for face liveness detection, the method comprising:
receiving a first image comprising a face of a user;
determining one or more two-dimensional (2D) facial landmark points based on the first image;
determining a three-dimensional (3D) pose of the face in the first image based on the one or more 2D facial landmark points and a first set of 3D facial landmark points, wherein the first set comprises one or more 3D facial landmark points in a 3D face model of the face of the user, and the one or more 3D facial landmark points of the first set correspond to the one or more 2D facial landmark points;
determining a homography mapping between the one or more 2D facial landmark points and a second set of 3D facial landmark points, wherein the second set comprises one or more other 3D facial landmark points that are perspectively projected from the one or more 3D facial landmark points of the first set based on the 3D pose; and
determining liveness of the face in the first image based on the homography mapping to determine actual presence of the user during capture of the first image.

16. The computer readable storage medium of claim 15 , wherein the method further comprises:
transforming the one or more 2D facial landmark points to one or more oth

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kolagunda Abhishek
• Wang Xiaolong
• Tasci Serafettin
• Li Dawei

Classifications

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V20/64 Primary

  Three-dimensional [3D] objects · CPC title