Three-dimensional facial reconstruction method and system

Patent metadata

Field	Value
Publication number	US-2017032565-A1
Application number	US-201515114649-A
Country	US
Kind code	A1
Filing date	Jul 13, 2015
Priority date	Jul 13, 2015
Publication date	Feb 2, 2017
Grant date	—

Abstract

The present invention is applicable to the field of image processing technology, provides a three-dimensional facial reconstruction method and system comprising: arranging three-dimensional imaging units with the same configuration on both of left side and right side of a target human face; implementing binocular calibration to the three-dimensional imaging units; establishing a polynomial relation between 3D point cloud coordinates captured by the three-dimensional imaging units and corresponding phases according to a result of the binocular calibration and determining the transformation relation among the 3D point cloud coordinates captured by two three-dimensional imaging units; capturing image sequences on the left side and right side of the target human face by the three-dimensional imaging units to obtain absolute phases of the image sequences; mapping the absolute phases of the image sequences to the 3D point cloud coordinates by using the polynomial relationship; unifying the 3D point cloud coordinates of the three-dimensional imaging units to a global coordinate system according to the transformation relationship. The present invention implements a rapid three-dimensional reconstruction of a face and improves the processing efficiency of three-dimensional facial reconstruction.

First claim

What is claimed is:

1 . A three-dimensional facial reconstruction method comprising:
arranging three-dimensional imaging units with the same configuration on left side and right side of a target human face; implementing binocular calibration to the three-dimensional imaging units, according to a result of the binocular calibration establishing a polynomial relation between 3D point cloud coordinates captured by the three-dimensional imaging units and corresponding phases and determining a transformation relation among the 3D point cloud coordinates captured by two three-dimensional imaging units; capturing image sequences on the left side and right side of the target human face by the three-dimensional imaging units to obtain absolute phases of the image sequences; mapping the absolute phases of the image sequences to the 3D point cloud coordinates by using the polynomial relationship; unifying the 3D point cloud coordinates of the three-dimensional imaging units to a global coordinate system according to the transformation relationship, to complete the three-dimensional reconstruction of the target human face.

2 . The method of claim 1 , wherein the step of arranging three-dimensional imaging units with the same configuration on left side and right side of a target human face comprises:
configuring a projector and a camera for each of the three-dimensional imaging unit, and using the projector as a reverse camera; providing a projection and capture control unit for controlling an image projection operation of the projector and an image capture operation of the camera.

3 . The method of claim 2 , wherein the step of implementing binocular calibration to the three-dimensional imaging units, establishing a polynomial relation between 3D point cloud coordinates captured by the three-dimensional imaging units and corresponding phases according to a result of the binocular calibration and determining the transformation relation among the 3D point cloud coordinates captured by two three-dimensional imaging units comprises:
based on a preset binocular imaging model, determining a point corresponding relationship between the position of the camera and the position of a projection chip of the projector and system parameters of each three-dimensional imaging unit; for a pixel positioned at any position, determining a ray emitted :from a optical center and through the pixel by the system parameters, and sampling N different 3D point cloud coordinates in a measuring range of the ray; according to the point corresponding relationship, projecting the 3D point cloud coordinates onto the projection chip, to obtain the corresponding phases of the 3D point cloud coordinates; and establishing the polynomial relation between the 3D point cloud coordinates captured by the three-dimensional imaging units and the corresponding phases.

4 . The method of claim 1 , wherein the step of determining the transformation relation among the 3D point cloud coordinates captured by two three-dimensional imaging units comprises:
determining the transformation relation as: where and are respectively a rotation matrix and a translation matrix of the three-dimensional imaging unit on the left and a world coordinate system, and are respectively the rotation matrix and translation matrix of the three-dimensional imaging unit on the right and the world coordinate system, and are respectively used to represent the transformation relationship between two three-dimensional imaging units.

5 . The method of claim 1 , wherein the method further comprises:
accelerating computing for parallel processing of each pixel in the mage sequences by using a graphics processing unit (GPU).

6 . A three-dimensional facial reconstruction system comprising:
an arrangement unit, configured to arranging three-dimensional imaging units with the same configuration on left side and right side of a target human face; a calibration unit, configured to implement binocular calibration to the three-dimensional imaging units, establish a polynomial relation between 3D point cloud coordinates captured by the three-dimensional imaging units and corresponding phases according to a result of the binocular calibration and determine the transformation relation among the 3D point cloud coordinates captured by two three-dimensional imaging units; a capture unit, configured to capture image sequences on the left side and right side of the target human face by the three-dimensional imaging units to obtain absolute phases of the image sequences; a mapping unit, configured to map the absolute phases of the image sequences to the 3D point cloud coordinates by using the polynomial relationship; a reconstruction unit, configured to unify the 3D point cloud coordinates of the three-dimensional imaging units to a global coordinate system according to the transformation relationship, to complete the three-dimensional reconstruction of the target human face.

7 . The system of claim 6 , wherein the arrangement unit comprises:
an arrangement subunit, configured to configure a projector and a camera for each of the three-dimensional imaging unit, and using the projector as a reverse camera; a setting subunit, configured to provide a projection and capture control unit for controlling an image projection operation of the projector and an image capture operation of the camera.

8 . The system of claim 7 , wherein the calibration unit comprises:
a determination subunit, configured to based on a preset binocular imaging model, determining a point corresponding relationship between the position of the camera and the position of a projection chip of the projector and system parameters of each three-dimensional imaging unit; a sampling subunit, configured to: for a pixel positioned at any position, determine a ray emitted from a optical center and through the pixel, and sample N different 3D point cloud coordinates in a measuring range of the ray; an establishing subunit, configured to according to the point corresponding relationship, project the 3D point cloud coordinates onto the projection chip, to obtain the corresponding phases of the 3D point cloud coordinates; and establish the polynomial relation between the 3D point cloud coordinates captured by the three-dimensional imaging units and the corresponding phases.

9 . The system of claim 6 , wherein the calibration unit further configured to:
determine the transformation relation as: where and are respectively a rotation matrix and a translation matrix of the three-dimensional imaging unit on the left and a world coordinate system, and are respectively the rotation matrix and translation matrix of the three-dimensional imaging unit on the right and the world coordinate system, and are respectively used to represent the transformation relationship two three-dimensional imaging units.

10 . The system of claim 6 , wherein the system further comprises:
a parallel computing unit, configured to accelerate computing for parallel processing of each pixel in the image sequences by using a graphics processing unit (GPU).

Assignees

• Univ Shenzhen

Inventors

• Peng Xiang
• Liu Xiaoli
• He Dong
• Chen Hailong
• Xu Chen

Classifications

• G06T15/205 Primary

  Image-based rendering · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T7/002

  Physics · mapped topic