Co-registration—simultaneous alignment and modeling of articulated 3D shapes

Patent metadata

Field	Value
Publication number	US-9898848-B2
Application number	US-201214433178-A
Country	US
Kind code	B2
Filing date	Dec 14, 2012
Priority date	Oct 5, 2012
Publication date	Feb 20, 2018
Grant date	Feb 20, 2018

Abstract

Present application refers to a method, a model generation unit and a computer program (product) for generating trained models (M) of moving persons, based on physically measured person scan data (S). The approach is based on a common template (T) for the respective person and on the measured person scan data (S) in different shapes and different poses. Scan data are measured with a 3D laser scanner. A generic personal model is used for co-registering a set of person scan data (S) aligning the template (T) to the set of person scans (S) while simultaneously training the generic personal model to become a trained person model (M) by constraining the generic person model to be scan-specific, person-specific and pose-specific and providing the trained model (M), based on the co registering of the measured object scan data (S).

First claim

The invention claimed is:

1. A model generation unit for generating deformable, non-rigid visual models (M) of physical objects, based on physically measured object scan data (S), comprising:
a template interface for providing at least one common template (T) for one of the physical objects;
a scanner for scanning said physical objects having respectively different shapes and poses to generate object scan data (S) that corresponds to physical landmarks on surfaces of said physical objects;
a database for storing at least one generic object model that corresponds to said object scan data;
an initializing interface for providing said object scan data (S) and said template data (T) in initialized form;
a co-registration unit for executing a non-linear objective function encompassing both a mesh alignment term and a model term for co-registering a set of ones of said object scan data (S) by executing registering and model generation in a combined manner, namely:
repeatedly a) aligning the template data (T) to the object scan data (S) to obtain aligned scans and training one of the models based on the scanned data, and b) constraining the aligning in step a) based on the one of the models (M) being trained; and

an output interface for generating said deformable, non-rigid visual models (M).

2. The model generation unit according to claim 1 , wherein aligning is executed by deforming the initialized template (T) to all initialized scans (S) of the set of initialized scans (S) in parallel and/or by inferring object shape from incomplete, noisy and/or ambiguous scan data.

3. The model generation unit according to claim 1 , wherein co-registration uses data present in another scan (S o ) in order to propagate information learned from the other scan (S o ) to present scan (S).

4. The model generation unit according to claim 1 , wherein at least some or all of the steps are executed iteratively so that the generic model may be replaced in the course of process with the trained model (M).

5. A model generation unit according to claim 1 , wherein aligning is done by applying a data penalty term for deforming the template (T) to match the scans (S) and by applying a data coupling term for constraining the deforming according to the trained model (M).

6. The model generation unit according to claim 1 , wherein the generic object model is a BlendSCAPE model, which is scan-specific, object-specific and pose-specific.

7. The model generation unit according to claim 1 , wherein a fit of an aligned template surface (T) to a surface of the initialized object scan (S) is evaluated by:

E
S

⁡

(

T
;
S

)

=

1

a
S

⁢

∫

x
s

∈
S

⁢

ρ
⁡

(

min

x
t

∈
T

⁢



x
s

-

x
t



)

.

8. The model generation unit according to claim 1 , wherein differences between the aligned template and the trained model are penalized by a coupling term, which is defined by:

E
C

⁡

(

T
,
θ
,
D
,
Q

)

=

∑
f

⁢

⁢

a
f

⁢



T
f

-

B
f

⁡

(
θ
)

⁢

D
f

⁢

Q
f

⁡

(
θ
)

⁢

T
f
*



F
2

.

9. The model generation unit according to claim 1 , wherein simple regularization terms are used to constrain object shape deformations (D) with regard to spatial smoothness and pose-dependent deformation model (Q).

10. The model generation unit according to clai

Assignees

• Max Planck Gesellschaft
• Max Planck Gesellschaft

Inventors

• Black Michael
• Hirshberg David L
• Loper Matthew
• Rachlin Eric
• Weiss Alex

Classifications

• G06T2213/04

  Animation description language · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T7/50

  Depth or shape recovery · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title