Sketch-based abstraction for character posing and synthesis

Patent metadata

Field	Value
Publication number	US-11087517-B2
Application number	US-201615172078-A
Country	US
Kind code	B2
Filing date	Jun 2, 2016
Priority date	Jun 3, 2015
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

In particular embodiments, a 2D representation of an object may be provided. A first method may comprise: receiving sketch input identifying a target position for a specified portion of the object; computing a deformation for the object within the context of a character rig specification for the object; and displaying an updated version of the object. A second method may comprise detecting sketch input; classifying the sketch input, based on the 2D representation, as an instantiation of the object; instantiating the object using a 3D model of the object; and displaying a 3D visual representation of the object.

First claim

What is claimed is:

1. A method, performed by one or more computing devices, comprising:
providing a two-dimensional (2D) representation of an object to be animated, wherein the 2D representation of the object is created by projection of a three-dimensional (3D) model of the object into a viewing plane;
mapping rig parameters of a character rig specification for the object to a 2D representation of the 3D model and to a surface mesh for the 3D model, wherein changes to the character rig specification simultaneously influence the surface mesh and the 2D representation;
receiving sketch input with respect to the object, wherein the sketch input identifies a target position for a specified portion of the object;
updating the character rig specification based on the target position for the specified portion of the object;
computing a deformation for the object based on the updated character rig specification, wherein:
the surface mesh and the 2D representation are deformed in accordance with the computed deformation,
the character rig specification comprises a defined space of deformations, and
the computed deformation is within the defined space; and

displaying an updated version of the 2D representation of the object based on the computed deformation.

2. The method of claim 1 , wherein the sketch input for the target position is drawn over the 2D representation of the object.

3. The method of claim 1 , wherein the deformation for the object is computed based on a nonlinear iterative closest point calculation between an original position of the specified portion and an updated position of the specified portion.

4. The method of claim 3 , wherein the updated position of the specified portion is determined by sampling the sketch input identifying the target position.

5. The method of claim 3 , wherein the nonlinear iterative closest point calculation incorporates correspondence weights determined between a first plurality of points along the 2D representation of the object and a second plurality of points along the sketch input identifying the target position.

6. The method of claim 3 , wherein the nonlinear iterative closest point calculation is based on minimizing an energy metric by incorporating aspects of the character rig specification as unknown values.

7. The method of claim 6 , wherein the minimizing the energy metric incorporates a regularization energy component.

8. The method of claim 1 , further comprising receiving initial sketch input comprising a curve drawn over a mesh for the object, wherein the 2D representation of the object is based on the initial sketch input, and wherein the curve is incorporated in the computed deformation.

9. The method of claim 8 , wherein the computing the deformation comprises penalizing any deformation in regions remote from the curve of the initial sketch input.

10. The method of claim 1 , wherein the 2D representation of the object is based on a set of curves embedded into the object, and wherein the curves are rigged alongside a mesh for the object.

11. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
provide a two-dimensional (2D) representation of an object to be animated, wherein the 2D representation of the object is created by projection of a three-dimensional (3D) model of the object into a viewing plane;
map rig parameters of a character rig specification for the object to a 2D representation of the 3D model and to a surface mesh for the 3D model, wherein changes to the character rig specification simultaneously influence the surface mesh and the 2D representation;
receive sketch input with respect to the object, wherein the sketch input identifies a target position for a specified portion of the object;
update the character rig specification based on the target position for the specified portion of the object;
compute a deformation for the object based on the updated character rig specification for the object, wherein the character rig specification comprises the 3D model of the object, and wherein:
the surface mesh and the 2D representation are deformed in accordance with the computed deformation,
the character rig specification comprises a defined space of deformations, and
the computed deformation is within the defined space; and

display an updated version of the 2D representation of the object based on the computed deformation.

12. The media of claim 11 , wherein the sketch input for the target position is drawn over the 2D representation of the object.

13. The media of claim 11 , wherein the deformation for the object is computed based on a nonlinear iterative closest point calculation between an original position of the specified portion and an updated position of the specified portion.

14. The media of claim 13 , wherein the updated position of the specified portion is determined by sampling the sketch input identifying the target position.

15. The media of claim 13 , wherein the nonlinear iterative closest point calculation incorporates correspondence weights determined between a first plurality of points along the 2D representation of the object and a second plurality of points along the sketch input identifying the target position.

16. The media of claim 13 , wherein the nonlinear iterative closest point calculation is based on minimizing an energy metric by incorporating aspects of the character rig specification as unknown values.

17. The media of claim 16 , wherein the minimizing the energy metric incorporates a regularization energy component.

18. The media of claim 11 , wherein the software is further operable when executed to receive initial sketch input comprising a curve drawn over a mesh for the object, wherein the 2D representation of the object is based on the initial sketch input, and wherein the curve is incorporated in the computed deformation.

19. The media of claim 18 , wherein the software that is operable when executed to compute the deformation comprises software that is operable when executed to penalize any deformation in regions remote from the curve of the initial sketch input.

20. A system comprising:
one or more processors; and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
provide a two-dimensional (2D) representation of an object to be animated, wherein the 2D representation of the object is created by projection of a three-dimensional (3D) model of the object into a viewing plane;

map rig parameters of a character rig specification for the object to a 2D representation of the 3D model and to a surface mesh for the 3D model, wherein changes to the character rig specification simultaneously influence the 3D model and the 2D representation of the 3D model;
receive sketch input with respect to the object, wherein the sketch input identifies a target position for a specified portion of the object;
update the character rig specification based on the target position for the specified portion of the object;
compute a deformation for the object based on the updated character rig specification for the object, wherein the character rig specification comprises the 3D model of the object, and wherein:
the surface mesh and the 2D representation are deformed in accordance with the computed deformation,
the character rig specification comprises a defined space of deformations, and
the computed d

Assignees

• Disney Entpr Inc
• Eth Zuerich Eidgenoessische Technische Hochschule Zuerich

Inventors

• Sumner Robert Walker
• Nitti Maurizio
• Coros Stelian
• Thomaszewski Bernhard
• Hahn Fabian Andreas
• Gross Markus
• Mutzel Frederik Rudolf

Classifications

• G06T11/23

  using straight lines or curves · CPC title

• G06T2219/2021

  Shape modification · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title