Techniques for detecting a three-dimensional face in facial recognition

Patent metadata

Field	Value
Publication number	US-11386707-B2
Application number	US-201916534435-A
Country	US
Kind code	B2
Filing date	Aug 7, 2019
Priority date	Aug 7, 2019
Publication date	Jul 12, 2022
Grant date	Jul 12, 2022

Abstract

Examples described herein generally relate to processing a first image captured by the first camera at a first time to determine a first set of multiple key points of a face, processing a second image captured by the second camera at a second time to determine a second set of the multiple key points on the face in the second image, determining a location of at least a portion of the multiple key points in a three-dimensional space based on a first location of each of the portion of the multiple key points in the first image and a second location of each of the portion of the multiple key points in the second image, and detecting whether the face is a valid three-dimensional face for facial recognition based at least in part on the location of at least the portion of the multiple key points in the three-dimensional space.

First claim

What is claimed is:

1. A system for detecting three-dimensional features in validating a face for facial recognition, comprising:
a first camera deployed at a first camera location and configured to capture images at a first position;
a second camera deployed at a second camera location and configured to capture images at a second position;
at least one processor configured to:
process a first image captured by the first camera at a first time to determine a first set of multiple key points of a face in the first image;
process a second image captured by the second camera at a second time, that is equal to or within a threshold time of the first time, to determine a second set of the multiple key points on the face in the second image;
determine, based on a first location of each of a portion of the multiple key points in the first image and a second location of each of the portion of the multiple key points in the second image, a three-dimensional location of at least the portion of the multiple key points in a three-dimensional space; and
detect whether the face is a valid three-dimensional face for facial recognition at least in part by comparing the three-dimensional location of at least the portion of the multiple key points to three-dimensional locations of similar key points of a classifier.

2. The system of claim 1 , wherein the at least one processor is configured to detect whether the face is a valid three-dimensional face for facial recognition at least in part by:
generating a confidence level regarding comparing the three-dimensional locations of at least the portion of the multiple key points to the three-dimensional locations similar key points of the classifier, wherein the classifier is trained using various facial images of different people; and
determining whether the confidence level achieves a threshold.

3. The system of claim 2 , wherein the at least one processor is further configured to apply, before comparing at least the portion of the multiple key points to similar key points of the classifier, at least one of an alignment or a scaling to the three-dimensional location of at least the portion of the multiple key points in the three-dimensional space.

4. The system of claim 1 , wherein the at least one processor is configured to determine the three-dimensional location of at least the portion of the multiple key points at least in part by performing triangulations for each of at least the portion of the multiple key points based on the first location of each of the portion of the multiple key points in the first image, the second location of each of the portion of the multiple key points in the second image, and pose information determined for the first camera and the second camera.

5. The system of claim 1 , wherein the at least one processor is configured to detect whether the face is a valid three-dimensional face for facial recognition at least in part by applying, to at least one of the first image or the second image, an infrared pass filter to filter out infrared light.

6. The system of claim 1 , wherein the first camera and the second camera each capture a video of multiple images over a period of time, and wherein the at least one processor is configured to detect whether the face is a valid three-dimensional face for facial recognition based at least in part on the three-dimensional location of at least the portion of the multiple key points determined over the multiple images.

7. The system of claim 1 , wherein the at least one processor is further configured to perform facial recognition based on detecting that the face is a valid three-dimensional face for facial recognition.

8. The system of claim 1 , wherein the at least one processor is configured to detect whether the face is a valid three-dimensional face for facial recognition at least in part by determining a confidence level associated with the detecting.

9. The system of claim 8 , wherein the at least one processor is further configured to provide the confidence level to an access control system for restricting, based at least in part on the confidence level, access to an area.

10. A computer-implemented method for detecting three-dimensional features in validating a face for facial recognition, comprising:
processing a first image captured by a first camera at a first time to determine a first set of multiple key points of a face in the first image;
processing a second image captured by a second camera at a second time, that is equal to or within a threshold time of the first time, to determine a second set of the multiple key points on the face in the second image;
determining, based on a first location of each of a portion of the multiple key points in the first image and a second location of each of the portion of the multiple key points in the second image, a three-dimensional location of at least the portion of the multiple key points in a three-dimensional space; and
detecting whether the face is a valid three-dimensional face for facial recognition at least in part by comparing the three-dimensional location of at least the portion of the multiple key points to three-dimensional locations of similar key points of a classifier.

11. The computer-implemented method of claim 10 , wherein detecting whether the face is a valid three-dimensional face for facial recognition comprises:
generating a confidence level regarding comparing three-dimensional locations of at least the portion of the multiple key points to the three-dimensional locations similar key points of the classifier, wherein the classifier is trained using various facial images of different people; and
determining whether the confidence level achieves a threshold.

12. The computer-implemented method of claim 11 , further comprising applying, before comparing at least the portion of the multiple key points to similar key points of the classifier, at least one of an alignment or a scaling to the three-dimensional location of at least the portion of the multiple key points in the three-dimensional space.

13. The computer-implemented method of claim 10 , wherein determining the three-dimensional location of at least the portion of the multiple key points comprises performing triangulations for each of at least the portion of the multiple key points based on the first location of each of the portion of the multiple key points in the first image, the second location of each of the portion of the multiple key points in the second image, and pose information determined for the first camera and the second camera.

14. The computer-implemented method of claim 10 , wherein detecting whether the face is a valid three-dimensional face for facial recognition includes applying, to at least one of the first image or the second image, an infrared pass filter to filter out infrared light.

15. The computer-implemented method of claim 10 , wherein the first camera and the second camera each capture a video of multiple images over a period of time, and wherein detecting whether the face is a valid three-dimensional face for facial recognition is based at least in part on the three-dimensional location of at least the portion of the multiple key points determined over the multiple images.

16. The computer-implemented method of claim 10 , further comprising performing facial recognition based on detecting that the face is a valid three-dimensional face for facial recognition.

17. The computer-implemented method of claim 10 , wherein detecting whether the face is a valid three-dimensional face for facial recognition includes determin

Assignees

• Johnson Controls Tyco IP Holdings LLP

Inventors

• Rozner Amit
• Kirsch Lior
• Boué Tamar
• Falik Yohay

Classifications

• G06V40/173 Primary

  face re-identification, e.g. recognising unknown faces across different face tracks · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T17/205

  Re-meshing · CPC title

• G06V40/166 Primary

  using acquisition arrangements · CPC title

• G06V20/647

  by matching two-dimensional images to three-dimensional objects · CPC title