Face comparison device, method, and recording medium

Patent metadata

Field	Value
Publication number	US-9916495-B2
Application number	US-201515127489-A
Country	US
Kind code	B2
Filing date	Mar 20, 2015
Priority date	Mar 28, 2014
Publication date	Mar 13, 2018
Grant date	Mar 13, 2018

Abstract

A face comparison device according to the present invention includes: an occluded-feature point estimation unit that estimates an occluded-feature point that is a feature point of an invisible face, and outputs position information of the occluded-feature point; a pseudo-feature point estimation unit that estimates a pseudo-feature point that is an erroneously detected feature point not captured within the input image due to occlusion, and outputs position information of the pseudo-feature point; and a comparison unit that generates a normalized image of the input image by using the position information of the occluded-feature point and the pseudo-feature point, generates a normalized image of an acquired comparison image, and compares the first facial feature vector extracted from the normalized image of the input image and the second facial feature vector extracted from the generated normalized image of the comparison image.

First claim

What is claimed is:

1. A face comparison device comprising:
an occluded-feature point estimation unit that estimates an occluded-feature point that is a feature point of an invisible face in an input image based on an angle of the face in the input image, a three-dimensional stereoscopic face model, and a model feature point arranged on the three-dimensional stereoscopic model, and outputs position information of the occluded-feature point;
a pseudo-feature point estimation unit that estimates a pseudo-feature point that is an erroneously detected feature point that is a feature point of the face not captured within the input image due to occlusion based on the angle of the face in the input image by using position information of the three-dimensional stereoscopic face model that is rotated, and outputs position information of the pseudo-feature point; and
a comparison unit that generates a normalized image of the input image by using the position information of the occluded-feature point and the position information of the pseudo-feature point, extracts a first facial feature vector from the generated normalized image of the input image, generates a normalized image of an acquired comparison image, extracts a second facial feature vector from the generated normalized image of the comparison image, and compares the first facial feature vector and the second facial feature vector.

2. The face comparison device according to claim 1 ,
wherein, when the occluded-feature point estimation unit rotates the three-dimensional stereoscopic face model and the model feature point arranged on the three-dimensional stereoscopic face model by using a same angle as the angle information of the face in the input image, and a line of sight that passes through the rotated model feature point intersects with the rotated three-dimensional stereoscopic face model at a position closer to an imaging device than the rotated model feature point, the occluded-feature point estimation unit estimates the intersection point as an occluded-feature point.

3. The face comparison device according to claim 1 ,
wherein, when the three-dimensional stereoscopic face model is orthographically projected, and a straight line that is parallel to a Z direction as a depth direction and passes through the model feature point that is rotated by the same angle as the angle information of the face in the input image intersects with the rotated three-dimensional stereoscopic face model at a position closer to an imaging device than the rotated model feature point, the occluded-feature point estimation unit estimates the intersection point as an occluded-feature point.

4. The face comparison device according to claim 1 ,
wherein the comparison unit corrects the position information of the feature point of the face in the input image by using, as a correction quantity, a quantity obtained by multiplying a difference between the position information of the occluded-feature point and the pseudo-feature point by a value of a difference between scales of the face in the input image and the rotated three-dimensional stereoscopic face model.

5. The face comparison device according to claim 1 ,
wherein the comparison unit uses, as a correction quantity, at least a quantity obtained by multiplying a difference between coordinate values of the occluded-feature point and the pseudo-feature point in an X direction as a horizontal direction by a value of a difference of scales between the face in the input image and the rotated three-dimensional stereoscopic face model as a scale factor.

6. The face comparison device according to claim 1 , further comprising:
a storage unit
that stores, in advance, the input image, the three-dimensional stereoscopic face model, the model feature point arranged on the three-dimensional stereoscopic face model, and the facial feature vector of the comparison image or the normalized image of the comparison image.

7. The face comparison device according to claim 1 , further comprising:
an angle estimation unit that calculates an angle of the face in the input image based on the feature point of the face in the input image.

8. A face comparison method comprising:
estimating an occluded-feature point that is a feature point of an invisible face in an input image based on an angle of the face in the input image, a three-dimensional stereoscopic face model, and a model feature point arranged on the three-dimensional stereoscopic face model, and outputting position information of the occluded-feature point;
estimating a pseudo-feature point that is an erroneously detected feature point that is a feature point of the face not captured within the input image due to occlusion based on the angle of the face in the input image by using position information of the three-dimensional stereoscopic face model that is rotated, and outputting position information of the pseudo-feature point; and
generating a normalized image of the input image by using the position information of the occluded-feature point and the position information of the pseudo-feature point, extracting a first facial feature vector from the generated normalized image of the input image, generating a normalized image of an acquired comparison image, extracting a second facial feature vector from the generated normalized image of the comparison image, and comparing the first facial feature vector and the second facial feature vector.

9. A computer readable non-transitory recording medium embodying a program, the program causing a computer to perform a method, the method comprising:
estimating an occluded-feature point that is a feature point of an invisible face in an input image based on an angle of the face in the input image, a three-dimensional stereoscopic face model, and a model feature point arranged on the three-dimensional stereoscopic face model, and outputting position information of the occluded-feature point;
estimating a pseudo-feature point that is an erroneously detected feature point that is a feature point of the face not captured within the input image due to occlusion based on the angle of the face in the input image by using position information of the three-dimensional stereoscopic face model that is rotated, and outputting position information of the pseudo-feature point; and
generating a normalized image of the input image by using the position information of the occluded-feature point and the position information of the pseudo-feature point, extracting a first facial feature vector from the generated normalized image of the input image, generating a normalized image of an acquired comparison image, extracting a second facial feature vector from the generated normalized image of the comparison image, and comparing the first facial feature vector and the second facial feature vector.

Assignees

• Nec Corp

Inventors

• Hayasaka Akihiro

Classifications

• G06V20/647 Primary

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06K9/00281 Primary

  Physics · mapped topic

• G06K9/00288

  Physics · mapped topic

• G06K9/00208

  Physics · mapped topic