Three-dimensional Human Face Reconstruction Method

Patent metadata

Field	Value
Publication number	US-2020167990-A1
Application number	US-201916578450-A
Country	US
Kind code	A1
Filing date	Sep 23, 2019
Priority date	Nov 27, 2018
Publication date	May 28, 2020
Grant date	—

Abstract

The invention is related to a method of three-dimensional face reconstruction by inputting a single face image to reconstruct a three-dimensional face model, therefore, the human face image is seen at various angles of three-dimensional face through rotating the model images.

First claim

What is claimed is:

1 . A method of three-dimensional human face reconstruction, comprising
inputting a two-dimensional face image; positioning said two-dimensional feature points for said two-dimensional face image, obtaining a plurality of two-dimensional feature point positions for said two-dimensional face image; converting said plurality of two-dimensional feature points into a plurality of three-dimensional coordinates, and converting said plurality of two-dimensional feature points into a corresponding said plurality of three-dimensional coordinates in accordance with an approximate computing, forming said plurality of three-dimensional coordinates to a first three-dimensional face model; finely tuning a three-dimensional face shape of said first three-dimensional face model, in order to obtain a second three-dimensional face model; compensating a face color of said second three-dimensional face model, in order to obtain a third three-dimensional face model; and outputting a three-dimensional face image in accordance with said third three-dimensional face model.

2 . The three-dimensional human face reconstruction method according to claim 1 , wherein the positioning method of the two-dimensional feature points comprises a neural network model.

3 . The three-dimensional human face reconstruction method according to claim 1 , wherein the three-dimensional face model comprises a color three-dimensional face model.

4 . The three-dimensional human face reconstruction method according to claim 1 , wherein upon converting the plurality of two-dimensional feature points into a plurality of three-dimensional coordinates and finely tuning a three-dimensional face shape of the first three-dimensional face model, the first three-dimensional face model adopting a three-dimensional face database to form the first three-dimensional face variable model through main component analysis, and the second three-dimensional face model adopts the three-dimensional face database to form the second three-dimensional face variable model through main component analysis.

5 . The three-dimensional human face reconstruction method according to claim 4 , wherein the first three-dimensional face variable model uses an average model as the basis, which adopting the linear combination of the plurality of feature templates to form a model differentiating from the first three-dimensional face model, wherein, the second three-dimensional face variable model using an average model as the basis, which adopts the linear combination of the plurality of feature templates to form a model differentiating from the second three-dimensional face model.

6 . The three-dimensional human face reconstruction method according to claim 5 , wherein the first three-dimensional face variable model and the second three-dimensional face model are multi-stage resolution three-dimensional face variable models, which being computed from lowest resolution to highest resolution, to obtain the first three-dimensional face variable model and the second three-dimensional face model.

7 . The three-dimensional human face reconstruction method according to claim 6 , wherein the multi-stage resolution three-dimensional face variable models comprises the three-dimensional face variable model from the first to above fifth stage resolution.

8 . The three-dimensional human face reconstruction method according to claim 1 , wherein upon carrying on the color compensation for the second three-dimensional face model, three arbitrary pointing on the second three-dimensional face model being used to form a face of a triangle, computing to see if the normal vector of every triangle faces outwards, in order to judge whether to fill the color on that face.

9 . The three-dimensional human face reconstruction method according to claim 8 , when the normal vector of the triangle faces outwards, it being judged that these three points being visible without been shielded, the second three-dimensional face model being projected to the two-dimensional plane in accordance with the coordinate of the two-dimensional plane to find out the corresponding color value, and patch back the color value to the second three-dimensional face model.

10 . The three-dimensional human face reconstruction method according to claim 8 , when the normal vector of the triangle faces inwards, it is judged that these three points being invisible with been shielded, and these three points being rear face.

11 . The three-dimensional human face reconstruction method according to claim 9 , after the color being filled, the average color value and the standard deviation being calculated, if the average value being over a predetermined value, wherein the corresponding color on the two-dimensional plane being not considered as the color of these three points, the interpolated color of surrounding color is filled.

12 . The three-dimensional human face reconstruction method according to claim 1 , wherein the steps for carrying on the fine tuning of the first three-dimensional face model of a three-dimensional face shape being adopted, the Newton method being adopted to get a first stage three-dimensional variable model projecting to a point on the two-dimensional plane, which being closest to a two-dimensional feature point for obtaining a three-dimensional rotation matrix, a two-dimensional deviation amount, a focus and a three-dimensional variable model parameter, to obtain a first stage three-dimensional face model.

13 . The three-dimensional human face reconstruction method according to claim 12 , wherein the steps for carrying on the fine tuning of the first three-dimensional face model of a three-dimensional face shape are adopted, when the resolution being the second stage, the three-dimensional rotation matrix, two-dimensional deviation amount, focus and three-dimensional variable model parameter being mapped to the second stage three-dimensional variable model, the second stage three-dimensional variable model being projected to the point on two-dimensional plane for computing a deviation value closest to the image profile point, the deviation value being adopted to adjust the three-dimensional variable model parameters, wherein the deviation value is less than the predetermined deviation value.

14 . The three-dimensional human face reconstruction method according to claim 12 , wherein the steps for carrying on the fine tuning of the first three-dimensional face model of a three-dimensional face shape being adopted, comprising when the resolution being above the second stage, wherein the three-dimensional rotation matrix, two-dimensional deviation amount, focus, deviation value of image profile point, position of two-dimensional feature points, deviation value of color projection and three-dimensional variable model parameter are mapped to a second stage three-dimensional variable model, to compute for projecting the second stage three-dimensional variable model to the point on two-dimensional plane.

15 . The three-dimensional human face reconstruction method according to claim 12 , wherein the steps for carrying on the fine tuning of the first three-dimensional face model of a three-dimensional face shape are adopted, comprising when the resolution being the Nth stage, the three-dimensional rotation matrix, two-dimensional deviation amount, focus and three-dimensional variable model parameter are serially mapped to a second stage three-dimensional variable model, wherein a second stage three-dimensional variable model until a (N−1)th three-dimensional variable model, the (N−1)th stage three-dimensional variable

Assignees

• Univ National Chiao Tung

Inventors

• Wu Bing-Fei
• Lin Chun-Hsien
• Wu Yi-Chiao
• Wu Bing-Jhang
• Huang Chih-Cheng
• Chung Meng-Liang

Classifications

• G06T15/205 Primary

  Image-based rendering · CPC title

• G06K9/00255

  Physics · mapped topic

• G06K9/00288

  Physics · mapped topic

• G06K9/00302

  Physics · mapped topic

• G06K2009/00328

  Physics · mapped topic