System and method for face recognition using three dimensions

Patent metadata

Field	Value
Publication number	US-9959455-B2
Application number	US-201715647238-A
Country	US
Kind code	B2
Filing date	Jul 11, 2017
Priority date	Jun 30, 2016
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

A system for facial recognition comprising at least one processor; at least one input operatively connected to the at least one processor; a database configured to store three-dimensional facial image data comprising facial feature coordinates in a predetermined common plane; the at least one processor configured to locate three-dimensional facial features in the image of the subject, estimate three-dimensional facial feature location coordinates in the image of the subject, obtain the three-dimensional facial feature location coordinates and orientation parameters in a coordinate system in which the facial features are located in the predetermined common plane; and compare the location of the facial feature coordinates of the subject to images of people in the database; whereby recognition, comparison and/or likeness of the facial images is determined by comparing the predetermined common plane facial feature coordinates of the subject to images in the database. A method is also disclosed.

First claim

The invention claimed is:

1. A method of facial recognition comprising:
inputting image data representing a plurality of images from a database; the database comprising images of people wherein the location of the three dimensional facial features is defined relative to a predetermined common plane;
inputting an image of a subject to be identified;
locating predetermined three-dimensional facial features in the image of the subject for comparison to the image data from the database;
estimating three-dimensional facial feature location coordinates of the subject head in the image of the subject;
obtaining the three-dimensional facial feature location coordinates and orientation parameters in a coordinate system in which the facial features are located in the predetermined common plane;
comparing the location of the coordinates of the subject to the locations of the coordinates of the images of people in the database relative to the predetermined common plane; and
determining the identity of the subject.

2. The method of claim 1 wherein the facial features are eyes, nose and mouth of a subject and wherein the coordinates of the location of the facial features are defined as the location of eye corners, nose tip and mouth corners; and wherein the predetermined common plane is a vertical plane passing the midpoints of the eye corners and mouth corners; and wherein the orientation parameters correlate to the yaw pitch and roll of the subject's head.

3. The method of claim 1 wherein the step of obtaining the three-dimensional facial feature location coordinates and orientation parameters comprises estimating the three-dimensional orientation of the subject's head; and wherein the predetermined common plane comprises the midpoints of the corners of the eyes and corners of the mouth.

4. The method of claim 2 wherein the yaw, pitch and roll of each head in the database of images of people is approximately the same since all facial features are specified relative to a predetermined common plane comprising the centers of the eye corners and mouth corners; and wherein the centers of the eye corners and mouth corners are located in a vertical plane.

5. The method of claim 1 wherein a plurality of geodesic distances between facial coordinates of the subject are determined and matched against the geodesic distances between facial coordinates of the image data in the database.

6. The method of claim 1 wherein the step of inputting an image of a subject to be identified comprises generating point clouds and surface meshes depicting the subject's head using at least one three-dimensional image sensor.

7. The method of claim 1 wherein the step of obtaining three-dimensional facial feature location coordinates and orientation parameters comprises estimating the three dimensional orientation of the subject's head and scale; and further comprises transforming the three-dimensional facial feature location coordinates of the corners of the eyes, nose tip, and corners of the mouth of the subject into corresponding coordinates relative to a vertical plane containing the midpoints of the corners of the eyes and corners of the mouth using a Gaussian Least Squares Differential Correction (GLSDC) minimization procedure.

8. The method of claim 1 wherein the predetermined common plane is a vertical plane comprising the midpoints of the corners of the eyes and corners of the mouth.

9. A method of facial recognition comprising:
for each image of a person in a database:
locating predetermined three-dimensional facial features in the image of the person;
estimating three-dimensional facial feature location coordinates;
obtaining the three-dimensional facial feature location coordinates and orientation parameters relative to a coordinate system defined by the location of facial features relative to the predetermined common plane;

inputting image data representing a plurality of images from the database;
inputting an image of a subject to be identified;
locating predetermined three-dimensional facial features in the image of the subject for comparison to the image data from the database;
estimating three-dimensional facial feature location coordinates of the subject head in the image of the subject;
obtaining the three-dimensional facial feature location coordinates and orientation parameters in a coordinate system in which the facial features are located in the predetermined common plane;
comparing the location of the coordinates of the subject to the locations of the coordinates of the images of people in the database; and
determining the identity of the subject.

10. The method of claim 9 wherein the step of obtaining the three-dimensional facial feature location coordinates and orientation parameters comprises using a Gaussian Least Squares Differential Correction (GLSDC) minimization procedure.

11. The method of claim 9 further comprising determining the geodesic distances between at least two of the facial features; and wherein the step of comparing the location of the coordinates of the subject to the locations of the coordinates of the images of people in the database comprises comparing the geodesic distances between at least two of the facial features.

12. The method of claim 1 wherein the inputting of an image of a subject to be identified comprising combining a plurality of images to form an image of the subject.

13. A system for facial recognition comprising:
at least one processor;
at least one input operatively connected to at least one processor and configured to input at least one image of a subject to be identified;
a database configured to store three-dimensional facial image data comprising facial feature coordinates in a predetermined common plane;
the at least one processor configured to locate predetermined three-dimensional facial features in the image of the subject, estimate three-dimensional facial feature location coordinates of the subject head in the image of the subject, and obtain the three-dimensional facial feature location coordinates and orientation parameters relative to a predetermined common plane; and compare the locations of the facial feature coordinates of the subject to the three-dimensional facial image data in the database relative to the predetermined common plane;
whereby recognition likeness of the facial images is determined by comparing the predetermined common plane facial feature coordinates of the subject to the predetermined common plane facial feature coordinates of the images in the database.

14. The system of claim 13 wherein the facial features are eyes, nose and mouth of a subject and wherein the facial features coordinates are the location of eye corners, nose tip and mouth corners; and wherein the predetermined common plane is a vertical plane containing the centers of the eye corners and mouth corners; and wherein orientation parameters correlate to the yaw pitch and roll of the subject's head.

15. The system of claim 13 wherein the predetermined common plane is a vertical plane containing centers of the corners of eyes and mouth corners.

16. The system of claim 13 wherein the database is configured to store a plurality of geodesic distances between facial feature coordinates of the images of people in the database and the at least one processor determines geodesic distances between facial feature coordinates of the subject and matches the geodesic distances against the geodesic distances between facial coordinates of the images of people in the database.

17. The system of claim 13 further comprising at least one three-d

Assignees

• U S Army Res Laboratory Attn Rdrl Loc I
• Us Army

Inventors

• Young Shiqiong Susan
• Ye Jinwei

Classifications

• G06K9/00228

  Physics · mapped topic

• G06K9/00268 Primary

  Physics · mapped topic

• G06K9/00288

  Physics · mapped topic

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V20/653 Primary

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title