Text-to-3D avatars

Patent metadata

Field	Value
Publication number	US-12340480-B2
Application number	US-202318155400-A
Country	US
Kind code	B2
Filing date	Jan 17, 2023
Priority date	Jan 17, 2023
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Three-dimensional (3D) avatars may be produced by stylizing a dataset of images based on a user-input text prompt input to a stable diffusion model, and using the output stylized dataset of images to train an efficient geometry-aware 3D generative adversarial network (EG3D) model.

First claim

What is claimed:

1. A method for producing a model to generate three-dimensional (3D) avatars, the method comprising:
stylizing a dataset of images by providing a user-input text prompt input and the dataset of images to a stable diffusion model; and
producing 3D avatars by providing the stylized dataset of images to an efficient geometry-aware 3D generative adversarial network (EG3D) model.

2. The method of claim 1 , wherein the user-input text prompt lists an illustrative style for the 3D avatars.

3. The method of claim 2 , wherein, along with the stylized dataset of images, a pose of respective ones of the stylized images is input to the EG3D model for training of the EG3D model.

4. The method of claim 3 , wherein the pose is provided as metadata corresponding to respective ones of the stylized images.

5. The method of claim 4 , wherein the pose includes Euler angles along x, y, and z axes.

6. The method of claim 3 , wherein the EG3D model generates the 3D avatars corresponding to respective ones of the stylized images in the illustrative style listed in the user-input text prompt.

7. The method of claim 3 , wherein the EG3D model generates the 3D avatars having a pitch, yaw, and roll corresponding to the pose provided for corresponding ones of the respective stylized images.

8. The method of claim 1 , wherein the stable diffusion model comprises a latent diffusion model.

9. A method for providing a selection of three-dimensional (3D) avatars for an application user, the method comprising:
providing a text prompt, received from an account for the application user, to a stable diffusion model;
inputting a dataset of images to the stable diffusion model;
stylizing the dataset of images, by the stable diffusion model, based on the text prompt;
training an efficient geometry-aware 3D generative adversarial network (EG3D) model with the stylized dataset of images; and
outputting 3D avatars from the EG3D model for selection by the application user.

10. The method of claim 9 , wherein the text prompt includes a desired style for the 3D avatars.

11. The method of claim 9 , wherein the images in the stylized dataset of images respectively include pose information.

12. The method of claim 11 , wherein the pose information is included as metadata and includes Euler angles along x, y, and z axis.

13. The method of claim 12 , wherein the application is a social media application.

14. The method of claim 12 , wherein the application is an online interactive game.

15. The method of claim 12 , wherein the dataset of images includes headshots.

16. A system for providing customized avatars, comprising:
an input unit to transmit a text prompt; and
a processing system to:
receive the text prompt,
receive a dataset of images,
stylize the dataset of images by applying a descriptor included in the text prompt to the dataset of images by executing a stable diffusion model,
train an image generating model with the stylized dataset of images,
generate avatars in a style corresponding to the descriptor included in the text prompt by executing the image generating model, and
output at least a portion of the generated avatars to a user device for selection.

17. The system of claim 16 , wherein the customized avatars are three-dimensional (3D).

18. The system of claim 17 , wherein the image generating model is an efficient geometry-aware 3D generative adversarial network (EG3D).

19. The system of claim 16 , wherein the descriptor is an illustration style.

20. The system of claim 16 , wherein stylized images included in the stylized dataset of images include corresponding pose information indicating Euler angles along x, y, and z axes.

21. The system of claim 20 , wherein the stylized images are headshots.

22. The system of claim 20 , wherein the processing system is to generate avatars corresponding to stylized images included in the stylized dataset of images utilizing the corresponding pose information.

23. Three-dimensional (3D) avatars that are generated in a style listed in a text prompt, the 3D avatars generated by:
receiving the text prompt from a user, the text prompt including a descriptor of an illustrative style for the 3D avatars;
applying the descriptor to a dataset of images by executing a stable diffusion model to modify the dataset of images in the illustrative style corresponding to the descriptor;
training an efficient geometry-aware 3D generative adversarial network (EG3D) model with the modified dataset of images; and
producing the 3D avatars corresponding to the modified dataset of images by executing the EG3D model.

24. The avatars of claim 23 , wherein images included in the modified dataset of images have respectively corresponding Euler angles representing a pose, and wherein the 3D avatars have a respective pitch, yaw, and roll corresponding to the Euler angles for the respective images.

Assignees

• Lemon Inc

Inventors

• Zhi Tiancheng
• Dudhat Rushikesh
• Liu Jing
• Luo Linjie

Classifications

• G06N3/094 Primary

  Adversarial learning · CPC title

• A63F13/52

  involving aspects of the displayed game scene · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06T2219/2024

  Style variation · CPC title

• G06N3/045

  Combinations of networks · CPC title