Method and apparatus for face recognition

Patent metadata

Field	Value
Publication number	US-10248844-B2
Application number	US-201615189454-A
Country	US
Kind code	B2
Filing date	Jun 22, 2016
Priority date	Aug 10, 2015
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

A training method of training an illumination compensation model includes extracting, from a training image, an albedo image of a face area, a surface normal image of the face area, and an illumination feature, the extracting being based on an illumination compensation model; generating an illumination restoration image based on the albedo image, the surface normal image, and the illumination feature; and training the illumination compensation model based on the training image and the illumination restoration image.

First claim

What is claimed is:

1. A face recognition method comprising:
inputting a first input image into an illumination compensation model and outputting, by the illumination compensation model, a first albedo image and a first surface normal image, the illumination compensation model being implemented by a neural network model;
inputting an enrolled image into the illumination compensation model and outputting, by the illumination compensation model, a second albedo image and a second surface normal image;
generating a first feature value by inputting the first albedo image and the first surface normal image extracted by the illumination compensation model to a face recognition model and outputting, by the face recognition model, the first feature value;
generating a second feature value by inputting the second albedo image and the second surface normal image extracted by the illumination compensation model to the face recognition model and outputting, by the face recognition model, the second feature value; and
determining a face recognition result based on the first and second feature values.

2. The method of claim 1 , wherein the determining comprises extracting an illumination component and an occlusion component from the first input image based on the illumination compensation model.

3. The method of claim 1 , wherein the illumination compensation model and the face recognition model are based on a convolutional neural network (CNN) model.

4. The method of claim 1 , further comprising:
outputting, from the face recognition model, an identification value corresponding to the first input image based on the first albedo image and the first surface normal image.

5. The method of claim 1 , wherein the first albedo image and the first surface normal image are independent of an illumination component comprised in the first input image.

6. The method of claim 1 , wherein the first albedo image indicates a texture component of a face area without regard to an illumination of the face area, and
the first surface normal image indicates a three-dimensional (3D) shape component of the face area without regard to the illumination.

7. A computer program embodied on a non-transitory computer readable medium, the computer program being configured to control a processor to perform the method of claim 1 .

8. A face recognition apparatus comprising:
a memory storing instructions; and
one or more processors configured to execute the instructions such that the one or more processors are configured to,
input a first input image into an illumination compensation model and outputting, by the illumination compensation model, a first albedo image and a first surface normal image, the illumination compensation model being implemented by a neural network model;
input an enrolled image into the illumination compensation model and outputting, by the illumination compensation model, a second albedo image and a second surface normal image;
generate a first feature value by inputting the first albedo image and the first surface normal image extracted by the illumination compensation model to a face recognition model and outputting, by the face recognition model, the first feature value;
generate a second feature value by inputting the second albedo image and the second surface normal image extracted by the illumination compensation model to the face recognition model and outputting, by the face recognition model, the second feature value; and
determine a face recognition result based on the first and second feature values.

9. The apparatus of claim 8 , wherein the one or more processors are configured to execute the instructions such that the one or more processors are configured to extract an illumination component and an occlusion component from the first input image based on the illumination compensation model.

10. The apparatus of claim 8 , wherein the illumination compensation model is based on a convolutional neural network (CNN) model applied to an encoder of an auto-encoder.

11. The apparatus of claim 8 , wherein the face recognition model is based on a convolutional neural network (CNN) model.

12. A training method of training an illumination compensation model, the method comprising:
extracting, from a training image, an albedo image of a face area, a surface normal image of the face area, and an illumination feature,
the extracting being based on an illumination compensation model;
generating an illumination restoration image based on the albedo image, the surface normal image, and the illumination feature; and
training the illumination compensation model based on the training image and the illumination restoration image by,
determining a loss function based on a difference between the training image and the illumination restoration image, and
updating parameters of the illumination compensation model based on the loss function.

13. The method of claim 12 , wherein the generating comprises:
generating the illumination restoration image by applying the albedo image, the surface normal image, and the illumination feature to a Lambertian model.

14. The method of claim 12 , wherein the training comprises:
updating a parameter of the illumination compensation model based on a difference between the training image and the illumination restoration image.

15. The method of claim 12 , wherein the extracting further includes extracting a mask image of the face area from the training image, based on the illumination compensation model.

16. The method of claim 15 , further comprising:
generating a mask restoration image based on the illumination restoration image and the mask image,
wherein the training includes training the illumination compensation model based on the training image and the mask restoration image.

Assignees

• Samsung Electronics Co Ltd
• Governing Council Univ Toronto

Inventors

• Kim Jungbae
• Salakhutdinov Ruslan
• Han Jaejoon
• Yoo Byungin

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/454 Primary

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V40/169

  Holistic features and representations, i.e. based on the facial image taken as a whole · CPC title

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title