Apparatus and method for creating three-dimensional personalized figure

Patent metadata

Field	Value
Publication number	US-9846804-B2
Application number	US-201514637344-A
Country	US
Kind code	B2
Filing date	Mar 3, 2015
Priority date	Mar 4, 2014
Publication date	Dec 19, 2017
Grant date	Dec 19, 2017

Abstract

Disclosed herein is an apparatus and method for automatically creating a 3D personalized figure suitable for 3D printing by detecting a face area and features for respective regions from face data acquired by heterogeneous sensors and by optimizing global/local transformation. The 3D personalized figure creation apparatus acquires face data of a user corresponding to a reconstruction target; extracts feature points for respective regions from the face data, and reconstructs unique 3D models of the user's face, based on the extracted feature points; creates 3D figure models based on the unique 3D models and previously stored facial expression models and body/adornment models; and verifies whether each 3D figure model has a structure and a shape corresponding to actual 3D printing, corrects and edits the 3D figure model based on results of verification, and outputs a 3D figure model corresponding to 3D printing.

First claim

What is claimed is:

1. A method for creating a three-dimensional (3D) personalized figure, the method being performed by an apparatus for creating a 3D personalized figure, comprising:
acquiring face data captured by a plurality of heterogeneous sensors of a user corresponding to a reconstruction target;
extracting feature points for respective regions from the face data, and reconstructing unique 3D models of the user's face, based on the extracted feature points;
creating 3D figure models based on the unique 3D models and previously stored facial expression models and body/adornment models by:
generating facial expressions that match an actually input facial expression of the user, based on the unique 3D models and the previously stored facial expression models; and
selecting a facial expression that stochastically matches a facial expression of the actual user from among the created facial expressions, and creating the 3D figure models based on the selected facial expression;
verifying whether each 3D figure model has a structure and a shape corresponding to actual 3D printing;
correcting and editing the 3D figure model based on results of the verification; and
outputting a 3D figure model corresponding to 3D printing,
wherein reconstructing the unique 3D models of the user's face comprises:
generating a front image based on the face data;
detecting a face area from the front image;
detecting feature points for respective regions from the face area;
detecting 3D corresponding points of the face data corresponding to the feature points of a 3D standard model, based on the feature points for respective regions, and then matching and transforming appearance information of the 3D standard model;
generating a face texture map using the transformed appearance information of the 3D standard model and the front image; and creating the unique 3D models of the user's face using the face texture map.

2. The method of claim 1 , the unique 3D models of the user's face comprises creating the unique 3D models of the user's face based on a procedure of causing a color of a remaining area of the face, which is not captured, to match a color of the face area, which is captured, using the face texture map.

3. The method of claim 1 , wherein creating the 3D figure models comprises:
creating adorned unique 3D models by combining the previously stored body/adornment models with the unique 3D models; and
selecting a body/adornment model that stochastically matches that of the actual user from among the adorned unique 3D models.

4. The method of claim 1 , wherein the face data corresponds to a 3D unrefined mesh model.

5. An apparatus for creating a 3D personalized figure, comprising:
an information acquisition unit for acquiring face data of a user corresponding to a reconstruction target from a plurality of heterogeneous sensors;
a face reconstruction unit for extracting feature points for respective regions from the face data, and reconstructing unique 3D models of the user's face, based on the extracted feature points;
a model creation unit for creating 3D figure models based on the unique 3D models and previously stored facial expression models and body/adornment models; and
a facial expression generation unit for generating facial expressions that match an actually input facial expression of the user, based on the unique 3D models and the previously stored facial expression models;
a selection unit for selecting a facial expression that stochastically matches that of the actual user from among the created facial expressions, and creates the 3D figure models based on the results of selection;
a model verification unit for verifying whether each 3D figure model has a structure and a shape corresponding to actual 3D printing, correcting and editing the 3D figure model based on results of verification, and outputting a 3D figure model corresponding to 3D printing,
wherein the face reconstruction unit comprises:
a generation unit for generating a front image based on the face data;
a face area detection unit for detecting a face area from the front image;
a feature point detection unit for detecting feature points for respective regions from the face area;
a matching and transformation unit for detecting 3D corresponding points of the face data corresponding to the feature points of a 3D standard model, based on the feature points for respective regions, and then matching and transforming appearance information of the 3D standard model;
a texture map generation unit for generating a face texture map using the transformed appearance information of the 3D standard model and the front image; and a unique 3D model creation unit for creating the unique 3D models of the user's face using the face texture map.

6. The apparatus of claim 5 , wherein the face data corresponds to a 3D unrefined mesh model.

7. The apparatus of claim 5 , wherein the unique 3D model creation unit creates the unique 3D models of the user's face based on a procedure of causing a color of a remaining area of the face, which is not captured, to match a color of the face area, which is captured, using the face texture map.

8. The apparatus of claim 5 , wherein the model creation unit comprises:
a body/adornment model creation unit for creating adorned unique 3D models by combining the previously stored body/adornment models with the unique 3D models;
wherein the selection unit selects a body/adornment model that stochastically matches that of the actual user from among the adorned unique 3D models.

9. The apparatus of claim 8 , wherein the model creation unit is configured such that a facial expression model storage unit including the facial expression models and a body/adornment model storage unit including the body/adornment models are operated in conjunction with each other.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Lim Seong-Jae
• Hwang Bon-Woo
• Kim Kap-Kee
• Yoon Seung-Uk
• Jun Hye-Ryeong
• Choi Jin-Sung
• Koo Bon-Ki

Classifications

• G05B19/40931 Primary

  concerning programming of geometry · CPC title

• G06V40/174

  Facial expression recognition · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• Y02P90/265

  Cross-Sectional Technologies · mapped topic