Resolving garment collisions using neural networks

Patent metadata

Field	Value
Publication number	US-11978144-B2
Application number	US-202217875081-A
Country	US
Kind code	B2
Filing date	Jul 27, 2022
Priority date	Jul 27, 2022
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

Embodiments are disclosed for using machine learning models to perform three-dimensional garment deformation due to character body motion with collision handling. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input, the input including character body shape parameters and character body pose parameters defining a character body, and garment parameters. The disclosed systems and methods further comprise generating, by a first neural network, a first set of garment vertices defining deformations of a garment with the character body based on the input. The disclosed systems and methods further comprise determining, by a second neural network, that the first set of garment vertices includes a second set of garment vertices penetrating the character body. The disclosed systems and methods further comprise modifying, by a third neural network, each garment vertex in the second set of garment vertices to positions outside the character body.

First claim

We claim:

1. A computer-implemented method, comprising:
receiving an input, the input including character body shape parameters, character body pose parameters, and garment parameters, the character body shape parameters and the character body pose parameters defining a character body;
generating, by a first neural network, a first set of garment vertices based on the input, the first set of garment vertices defining deformations of a garment with the character body;
determining, by a second neural network, that the first set of garment vertices includes a second set of garment vertices penetrating the character body; and
modifying, by a third neural network, each garment vertex in the second set of garment vertices to positions outside the character body.

2. The computer-implemented method of claim 1 , wherein determining that the first set of garment vertices includes the second set of garment vertices penetrating the character body comprises:
processing, by the second neural network, the first set of garment vertices to determine distance values for each garment vertex of the first set of garment vertices to a closest point on a surface of the character body; and
for each garment vertex of the first set of garment vertices, determining that the garment vertex is in the second set of garment vertices when a distance value of the garment vertex to the closest point on the surface of the character body indicates that the garment vertex is inside the character body.

3. The computer-implemented method of claim 2 , wherein the distance value of the garment vertex to the closest point on the surface of the character body indicates that the garment vertex is inside the character body when the distance value is a negative value.

4. The computer-implemented method of claim 2 , further comprising:
for each garment vertex of the second set of garment vertices, determining a gradient of the determined distance value.

5. The computer-implemented method of claim 2 , wherein modifying each garment vertex in the second set of garment vertices to the positions outside the character body comprises:
for each garment vertex of the second set of garment vertices:
predicting an offset distance along a direction of a gradient of the distance value associated with the corresponding garment vertex, and
modifying a location of the garment vertex from an initial location to an updated location based on the predicted offset distance.

6. The computer-implemented method of claim 5 , wherein predicting the offset distance along the direction of the gradient of the distance value associated with the corresponding garment vertex comprises:
processing, by the third neural network, a feature vector representing the input, the first set of garment vertices, the distance value of each garment vertex in the second set of garment vertices, and a gradient of the distance value of each garment vertex in the second set of garment vertices.

7. The computer-implemented method of claim 1 , further comprising:
generating an updated set of garment vertices including the first set of garment vertices not in the second set of garment vertices and the modified second set of garment vertices.

8. A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
receiving an input, the input including character body shape parameters, character body pose parameters, and garment parameters, the character body shape parameters and the character body pose parameters defining a character body;
generating, by a first neural network, a first set of garment vertices based on the input, the first set of garment vertices defining deformations of a garment with the character body;
determining, by a second neural network, that the first set of garment vertices includes a second set of garment vertices penetrating the character body; and
modifying, by a third neural network, each garment vertex in the second set of garment vertices to positions outside the character body.

9. The non-transitory computer-readable medium of claim 8 , wherein to determine that the first set of garment vertices includes the second set of garment vertices penetrating the character body, the instructions further cause the processing device to perform operations comprising:
processing, by the second neural network, the first set of garment vertices to determine distance values for each garment vertex of the first set of garment vertices to a closest point on a surface of the character body; and
for each garment vertex of the first set of garment vertices, determining that the garment vertex is in the second set of garment vertices when a distance value of the garment vertex to the closest point on the surface of the character body indicates that the garment vertex is inside the character body.

10. The non-transitory computer-readable medium of claim 9 , wherein the distance value of the garment vertex to the closest point on the surface of the character body indicates that the garment vertex is inside the character body when the distance value is a negative value.

11. The non-transitory computer-readable medium of claim 9 , wherein the instructions further cause the processing device to perform operations comprising:
for each garment vertex of the second set of garment vertices, determining a gradient of the determined distance value.

12. The non-transitory computer-readable medium of claim 9 , wherein to modify each garment vertex in the second set of garment vertices to the positions outside the character body, the instructions further cause the processing device to perform operations comprising:
for each garment vertex of the second set of garment vertices:
predicting an offset distance along a direction of a gradient of the distance value associated with the corresponding garment vertex, and
modifying a location of the garment vertex from an initial location to an updated location based on the predicted offset distance.

13. The non-transitory computer-readable medium of claim 12 , wherein to predict the offset distance along the direction of the gradient of the distance value associated with the corresponding garment vertex, the instructions further cause the processing device to perform operations comprising:
processing, by the third neural network, a feature vector representing the input, the first set of garment vertices, the distance value of each garment vertex in the second set of garment vertices, and a gradient of the distance value of each garment vertex in the second set of garment vertices.

14. The non-transitory computer-readable medium of claim 9 , wherein the instructions further cause the processing device to perform operations comprising:
generating an updated set of garment vertices including the first set of garment vertices not in the second set of garment vertices and the modified second set of garment vertices.

15. A system comprising:
a memory component; and
a processing device coupled to the memory component, the processing device to perform operations comprising:
receiving an input, the input including character body shape parameters, character body pose parameters, and garment parameters, the character body shape parameters and the character body pose parameters defining a character body;
generating, by a first neural network, a first set of garment vertices based on the input, the first set of garment vertices defining deformations of a garment with the character body;
determining, by a second neural network, that the first

Assignees

• Adobe Inc

Inventors

• Zhou Yi
• Wang Yangtuanfeng
• Sun Xin
• Tan Qingyang
• Ceylan Aksit Duygu

Classifications

• G06T13/40 Primary

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T2210/16

  Cloth · CPC title

• G06T2210/21

  Collision detection, intersection · CPC title

• G06F30/27 Primary

  using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title