Method and system using a data-driven model for monocular face tracking

Patent metadata

Field	Value
Publication number	US-9400921-B2
Application number	US-85239801-A
Country	US
Kind code	B2
Filing date	May 9, 2001
Priority date	May 9, 2001
Publication date	Jul 26, 2016
Grant date	Jul 26, 2016

Abstract

A method and system using a data-driven model for monocular face tracking are disclosed, which provide a versatile system for tracking three-dimensional (3D) images, e.g., a face, using a single camera. For one method, stereo data based on input image sequences is obtained. A 3D model is built using the obtained stereo data. A monocular image sequence is tracked using the built 3D model. Principal Component Analysis (PCA) can be applied to the stereo data to learn, e.g., possible facial deformations, and to build a data-driven 3D model (“3D face model”). The 3D face model can be used to approximate a generic shape (e.g., facial pose) as a linear combination of shape basis vectors based on the PCA analysis.

First claim

What is claimed is:

1. A method for image processing comprising:
obtaining stereo data based on input image sequences from of varying facial expressions:
building a three-dimensional (3D) model using the obtained stereo data to obtain principal shape vectors; and
tracking a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

2. The method of claim 1 , wherein the building of the 3D model includes processing the obtained stereo data using a Principal Component Analysis (PCA).

3. The method of claim 2 , wherein the processed stereo data using PCA allows the 3D model to approximate a generic shape as the linear combination of the shape basis vectors.

4. The method of claim 1 , wherein the tracking of the monocular image sequence includes tracking of a monocular image sequence of facial deformations using the built 3D model.

5. A computing system comprising:
an input unit to stereo data based on input image sequences from of varying facial expressions; and
a processing unit to build a three-dimensional (3D) model using the obtained stereo data to approximate a generic shape as a linear combination of shape basis vectors and track a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

6. The computing system of claim 5 , wherein the processing unit is to process the obtained stereo data using a Principal Component Analysis (PCA).

7. The computing system of claim 6 , wherein the processed stereo data using PCA allows the 3D model to approximate a generic shape as the linear combination of the shape basis vectors.

8. The computing system of claim 5 , wherein the processing unit is to track a monocular image sequence of facial deformations using the built 3D model.

9. A non-transitory machine-readable medium providing instructions, which if executed by a processor, causes the processor to perform an operation comprising:
obtaining stereo data based on input image sequences from of varying facial expressions:
building a three-dimensional (3D) model using the obtained stereo data to approximate a generic shape as a linear combination of shape basis vectors; and
tracking a second input image sequence using the 3D model to approximate a linear combination of the principal shape vectors of a facial expression in the second input image sequence, wherein the second input image sequence is a monocular image sequence.

10. The machine-readable medium of claim 9 , further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising: processing the obtained stereo data using a Principal Component Analysis (PCA).

11. The machine-readable medium of claim 10 , further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising:
approximate a generic shape as the linear combination of the shape basis vectors based on the processed stereo data using PCA.

12. The machine-readable medium of claim 9 , further providing instructions, which if executed by the processor, causes the processor to perform an operation comprising: tracking of a monocular image sequence of facial deformations using the built 3D model.

Assignees

• Bouguet Jean-Yves
• Grzeszczuk Radek
• Gokturk Salih
• Intel Corp

Inventors

• Bouguet Jean-Yves
• Grzeszczuk Radek
• Gokturk Salih

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/251

  involving models · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/189

  Recording image signals; Reproducing recorded image signals · CPC title