Methods, systems, and computer readable media for modeling garments using single view images

Patent metadata

Field	Value
Publication number	US-10546433-B2
Application number	US-201816054599-A
Country	US
Kind code	B2
Filing date	Aug 3, 2018
Priority date	Aug 3, 2017
Publication date	Jan 28, 2020
Grant date	Jan 28, 2020

Abstract

Methods, systems, and computer readable media for modeling garments using single view images are disclosed. According to one method, the method includes receiving an image depicting a person wearing at least one garment. The method also includes constructing a body model based on the person in the image and a template from a body model database. The method further includes constructing at least one garment model based on the at least one garment in the image and at least one template from a garment model database. The method also includes constructing a combined model based on the body model and the at least one garment model. The method further includes adjusting the combined model by modifying body pose parameters and determining garment material properties and sizing parameters.

First claim

What is claimed is:

1. A method for modeling garments using single view images, the method comprising:
receiving an image depicting a person wearing at least one garment;
constructing a body model based on the person in the image and a template from a body model database;
constructing at least one garment model based on the at least one garment in the image and at least one template from a garment model database, wherein constructing the at least one garment model includes computing at least one average wrinkle density for the at least one garment and using the at least one average wrinkle density and a physics-based cloth simulation to derive garment material properties of the at least one garment model, wherein using the at least one average wrinkle density and a physics-based cloth simulation to derive the garment material properties of the at least one garment model includes determining a first reference garment based on a first average wrinkle density and using detected garment information about the first garment from the image and known garment material properties associated with the first reference garment to derive the garment material properties of the at least one garment model;
constructing a combined model based on the body model and the at least one garment model; and
adjusting the combined model by modifying body pose parameters and the garment material properties and sizing parameters.

2. The method of claim 1 comprising:
prior to constructing the body model:
receiving a body outline associated with the person in the image; and
identifying multiple human body joint locations in the image.

3. The method of claim 2 wherein the body outline or the multiple human body joint locations is provided by a user.

4. The method of claim 2 wherein the body outline or the multiple human body joint locations is determined automatically using image processing.

5. The method of claim 1 wherein constructing the body model includes determining a human body shape and a body pose for the image.

6. The method of claim 1 wherein constructing the at least one garment model includes estimating locations and types of garments in the image, using a database of garment patterns and/or templates.

7. The method of claim 1 wherein constructing the combined model includes aligning the at least one garment model and the body model based on joint related information for the body model and the at least one garment model.

8. The method of claim 1 wherein adjusting the combined model includes using at least one iterative process and the physics-based cloth simulation for adjusting garment sizing information, the garment material properties, and the body pose parameters.

9. A system for modeling garments using single view images, the system comprising:
at least one processor; and
a garment recovery module (GRM) implemented using the at least one processor, the GRM is configured for:
receiving an image depicting a person wearing at least one garment;
constructing a body model based on the person in the image and a template from a body model database;
constructing at least one garment model based on the at least one garment in the image and at least one template from a garment model database, wherein constructing the at least one garment model includes computing at least one average wrinkle density for the at least one garment and using the at least one average wrinkle density and a physics-based cloth simulation to derive garment material properties of the at least one garment model, wherein using the at least one average wrinkle density and a physics-based cloth simulation to derive the garment material properties of the at least one garment includes determining a first reference garment based on a first average wrinkle density and using detected garment information about the first garment from the image and known garment material properties associated with the first reference garment to derive the garment material properties of the at least one garment model;
constructing a combined model based on the body model and the at least one garment model; and
adjusting the combined model by modifying body pose parameters and the garment material properties and sizing parameters.

10. The system of claim 9 wherein the GRM is configured for:
prior to constructing the body model:
receiving a body outline associated with the person in the image; and
identifying multiple human body joint locations in the image.

11. The system of claim 10 wherein the body outline or the multiple human body joint locations is provided by a user.

12. The system of claim 10 wherein the body outline or the multiple human body joint locations is determined automatically using image processing.

13. The system of claim 9 wherein the GRM is configured for determining a human body shape and a body pose for the image.

14. The system of claim 9 wherein the GRM is configured for estimating locations and types of garments in the image.

15. The system of claim 9 wherein the GRM is configured for aligning the at least one garment model and the body model based on joint related information for the body model and the at least one garment model.

16. The system of claim 9 wherein the GRM is configured for at least one iterative process and the physics-based cloth simulation for adjusting and calculating garment sizing information, the garment material properties, and the body pose parameters.

17. A non-transitory computer readable medium storing executable instructions that, when executed by one or more computers cause the one or more computers to perform operations comprising:
receiving an image depicting a person wearing at least one garment;
constructing a body model based on the person in the image and a template from a body model database;
constructing at least one garment model based on the at least one garment in the image and at least one template from a garment model database;
constructing a combined model based on the body model and the at least one garment model, wherein constructing the at least one garment model includes computing at least one average wrinkle density for the at least one garment and using the at least one average wrinkle density and a physics-based cloth simulation to derive garment material properties of the at least one garment model, wherein using the at least one average wrinkle density and a physics-based cloth simulation to derive the garment material properties of the at least one garment model includes determining a first reference garment based on a first average wrinkle density and using detected garment information about the first garment from the image and known garment material properties associated with the first reference garment to derive the garment material properties of the at least one garment model; and
adjusting the combined model by modifying body pose parameters and the garment material properties and sizing parameters.

18. The non-transitory computer readable medium of claim 17 , the operations comprising:
prior to constructing the body model:
receiving body outline associated with the person in the image; and
identifying multiple human body joint locations in the image.

Assignees

• Univ North Carolina Chapel Hill

Inventors

• Lin Ming C
• Yang Shan

Classifications

• G06T2210/16

  Cloth · CPC title

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title

• G06T7/62

  of area, perimeter, diameter or volume · CPC title

• G06T17/205

  Re-meshing · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title