Facial animation models

Patent metadata

Field	Value
Publication number	US-9747716-B1
Application number	US-201313839243-A
Country	US
Kind code	B1
Filing date	Mar 15, 2013
Priority date	Mar 15, 2013
Publication date	Aug 29, 2017
Grant date	Aug 29, 2017

Abstract

A system includes a computing device that includes a memory configured to store instructions. The system also includes a processor configured to execute the instructions to perform a method that includes receiving multiple representations of an object. Each of the representations includes position information of the object and corresponds to an instance in time. For at least one of the representations, the method includes defining a contour that represents a movable silhouette of a surface feature of the object. The method also includes producing a deformable model of the surface of the object from the defined contour and from the at least one representation of the object.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, at a processor, multiple images capturing an actor's facial performance at different instances in time;
from each of the images, estimating, by the processor, position information indicating positions of a set of features on the actor's face, wherein the set of features include a first curve in a mouth region of the actor;
for each of the images, defining, by the processor, based on the position information estimated from the image, a silhouette contour that represents a movable boundary between a visual portion and a hidden portion of the actor's mouth, wherein the silhouette contour is defined such that the silhouette contour corresponds to the first curve;
producing, at the processor, one or more deformable models representing a surface of the actor's face at the different instances in time based on the position information estimated from the images;
generating, by the processor, one or more corrective shapes based on the silhouette contour and correspondence between the silhouette contour and the first curve defined in the images;
defining separation thresholds for closing the portion of the deformable model; and
applying, by the processor, the corrective shapes to the one or more deformable models to adjust the mouth region of the one or more deformable models, wherein applying the corrective shapes to the one or more deformable models to adjust the mouth region of the one or more deformable models includes:
adjusting an inner lip shape or a corner of the mouth region of the one or more deformable models of the animated character including closing the portion of the deformable model based on the separation thresholds.

2. The computer-implemented method of claim 1 , wherein defining the silhouette contour includes projecting, by the processor, a series of vertices representing end points of the silhouette contour to corresponding end points of the first curve.

3. The computer-implemented method of claim 1 , wherein the set of features on the actor's face include a second curve in the mouth region of the actor, wherein the silhouette contour is defined such that the silhouette contour further corresponds to the second curve, and wherein the one or more corrective shapes are generated further based on correspondence between the silhouette contour and the second curve.

4. The computer-implemented method of claim 1 , wherein the first curve is on a lower lip or on a upper lip of the actor.

5. The computer-implemented method of claim 1 , wherein the one or more deformable models include a blendshape representing a base facial expression by the actor.

6. A computer-implemented method comprising:
receiving multiple images of an actor's mouth for generating a deformable model representing an animated character, the multiple images being captured while a dialogue is being delivered by the actor;
from each of the images, estimating, by the processor, position information indicating positions of a set of features on the actor's face, wherein the set of features include a first curve in a mouth region of the actor;
for each of the images, defining, based on the position information estimated from the image, a silhouette contour that represents a movable boundary of the actor's mouth, wherein the silhouette contour is defined such that the silhouette contour corresponds to the first curve;
defining separation thresholds for closing the portion of the deformable model;
based on the silhouette contour, and correspondence between the silhouette contour and the first curve defined in the images, adjusting the geometry of a portion of a deformable model of the animated character to reduce the size of a portion of a mouth of the animated character including closing the portion of the deformable model based on the separation thresholds, wherein adjusting the geometry of the portion of the deformable model of the animated character to reduce the size of the portion of the mouth of the animated character includes adjusting an inner lip shape or a corner of the mouth of the deformable model of the animated character; and
generating an image of the animated character that presents the adjusted portions of the animated character's mouth.

7. A system comprising:
a computing device comprising:
a memory configured to store instructions; and
a processor configured to execute the instructions to perform a method comprising:
receiving, at a processor, multiple images capturing an actor's facial performance at different instances in time;
from each of the images, estimating, by the processor, position information indicating positions of a set of features on the actor's face, wherein the set of features include a first curve in a mouth region of the actor;
for each of the images, defining, by the processor, based on the position information estimated from the image, a silhouette contour that represents a movable boundary between a visual portion and a hidden portion of the actor's mouth, wherein the silhouette contour is defined such that the silhouette contour corresponds to the first curve;
producing one or more deformable models representing a surface of the actor's face at the different instances in time based on the position information estimated from the images;
generating one or more corrective shapes based on the silhouette contour and correspondence between the silhouette contour and the first curve defined in the images;
defining separation thresholds for closing the portion of the deformable model; and

applying the corrective shapes to the one or more deformable models to adjust the mouth region of the one or more deformable models based on thresholds, wherein applying the corrective shapes to the one or more deformable models to adjust the mouth region of the one or more deformable models includes:
adjusting an inner lip shape or a corner of the mouth region of the one or more deformable models of the animated character including closing the portion of the deformable model based on the separation thresholds.

8. The system of claim 7 , wherein defining the silhouette contour includes projecting, by the processor, a series of vertices representing end points of the silhouette contour to corresponding end points of the first curve.

9. The system of claim 7 , wherein the set of features on the actor's face include a second curve in the mouth region of the actor, wherein the silhouette contour is defined such that the silhouette contour further corresponds to the second curve, and wherein the one or more corrective shapes are generated further based on correspondence between the silhouette contour and the second curve.

10. The system of claim 7 , wherein the first curve is on a lower lip or on a upper lip of the actor.

11. The system of claim 7 , wherein the one or more deformable models include a blendshape representing a base facial expression by the actor.

12. A system comprising:
a computing device comprising:
a memory configured to store instructions; and
a processor configured to execute the instructions to perform a method comprising:
receiving multiple images an actor's mouth for generating a deformable model representing an animated character, the multiple images being captured while a dialogue is being delivered by the actor;
from each of the images, estimating, by the processor, position information indicating positions of a set of features on the actor's face, wherein the set of features include a first curve in a mouth region of the actor;
for each of the images, defining, based on the position information estimated from the image, a silhouette contour that represents a movable boun

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Mallet Ronald
• Ye Yuting
• Koperwas Michael
• Goldenthal Adrian R
• Bhat Kiran S

Classifications

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2021

  Shape modification · CPC title