Method and apparatus for generating virtual character, electronic device and readable storage medium

Patent metadata

Field	Value
Publication number	US-12272002-B2
Application number	US-202218082997-A
Country	US
Kind code	B2
Filing date	Dec 16, 2022
Priority date	Jun 24, 2022
Publication date	Apr 8, 2025
Grant date	Apr 8, 2025

Abstract

A method and apparatus for generating a virtual character, an electronic device and a computer readable storage medium are provided. The method includes: performing mesh simplification on an initial model of a virtual character to obtain a mesh-simplified model; obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.

First claim

What is claimed is:

1. A method for generating a virtual character, the method comprising:
performing mesh simplification on an initial model of the virtual character by using a standard mesh simplification algorithm with a mesh simplification mark, to obtain a mesh-simplified model; wherein the mesh simplification mark is used to indicate that a degree of mesh simplification on a facial area of the initial model is less than a degree of mesh simplification on other areas;
obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and
establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.

2. The method according to claim 1 , wherein the method further comprises:
synchronizing and updating coordinates of the mesh-simplified initial model in a dimension of a two-dimensional map, to obtain an updated model; and
performing protective processing on areas of five facial sense organs of the updated model by using skeleton skin information of the initial model.

3. The method according to claim 1 , wherein the method further comprises:
determining a device type of a current device; and
determining, in response to the device type being a mobile terminal, a parameter of a target mesh simplification degree matching a computing performance of the current mobile terminal.

4. The method according to claim 3 , wherein the performing mesh simplification on the initial model by using the standard mesh simplification algorithm with the mesh simplification mark, comprises:
performing mesh simplification on the initial model by using the standard mesh simplification algorithm with a facial boundary protection mark, wherein a parameter of a mesh simplification degree of the standard mesh simplification algorithm is the parameter of the target mesh simplification degree.

5. The method according to claim 1 , wherein the method further comprises:
performing a model alignment between the mesh-simplified model and the initial model, and updating coordinates of two-dimensional maps of the mesh-simplified model and the initial model.

6. The method according to claim 1 , wherein the obtaining the first target model by performing white model mapping rendering on the area of each material type on the mesh-simplified model, and obtaining the second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model, comprises:
performing area partitioning on the mesh-simplified model according to material type, to obtain a partition model;
establishing a mapping relationship between a two-dimensional map of an area of each material type and the white model mapping rendering by using a differentiable renderer and a white model mapping rendering technology, to obtain the first target model; and
establishing a mapping relationship between the two-dimensional map of an area of each material type and the hyper-realistic rendering by using the differentiable renderer and a hyper-realistic pipeline, to obtain the second target model.

7. The method according to claim 1 , wherein the establishing the bidirectional mapping between the first target model and the second target model, and obtaining the target virtual character through iterative updating of the bidirectional mapping, comprises:
establishing a first mapping relationship between the first target model and the second target model by using a recurrent neural network;
establishing a second mapping relationship between the second target model and the first target model by using the recurrent neural network;
performing iterative updating on the first mapping relationship and the second mapping relationship, based on a perceptual loss function and an adversarial loss function in the recurrent neural network, to obtain an updated first mapping relationship and an updated second mapping relationship; and
updating the second target model by using the updated first mapping relationship and the updated second mapping relationship, to obtain the target virtual character.

8. The method according to claim 1 , wherein the method further comprises:
acquiring a user image of a target user; and
generating a virtual character corresponding to the target user in a virtual data space, based on the user image and the target virtual character.

9. An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
performing mesh simplification on an initial model of the virtual character by using a standard mesh simplification algorithm with a mesh simplification mark, to obtain a mesh-simplified model; wherein the mesh simplification mark is used to indicate that a degree of mesh simplification on a facial area of the initial model is less than a degree of mesh simplification on other areas;
obtaining a first target model by performing white model mapping rendering on an area of each material type on the mesh-simplified model, and obtaining a second target model by performing hyper-realistic rendering on the area of each material type on the mesh-simplified model; and
establishing a bidirectional mapping between the first target model and the second target model, and obtaining a target virtual character through iterative updating of the bidirectional mapping.

10. The electronic device according to claim 9 , wherein the operations further comprise:
synchronizing and updating coordinates of the mesh-simplified initial model in a dimension of a two-dimensional map, to obtain an updated model; and
performing protective processing on areas of five facial sense organs of the updated model by using skeleton skin information of the initial model.

11. The electronic device according to claim 9 , wherein the operations further comprise:
determining a device type of a current device; and
determining, in response to the device type being a mobile terminal, a parameter of a target mesh simplification degree matching a computing performance of the current mobile terminal.

12. The electronic device according to claim 11 , wherein the performing mesh simplification on the initial model by using the standard mesh simplification algorithm with the mesh simplification mark, comprises:
performing mesh simplification on the initial model by using the standard mesh simplification algorithm with a facial boundary protection mark, wherein a parameter of a mesh simplification degree of the standard mesh simplification algorithm is the parameter of the target mesh simplification degree.

13. The electronic device according to claim 9 , wherein the operations further comprise:
performing a model alignment between the mesh-simplified model and the initial model, and updating coordinates of two-dimensional maps of the mesh-simplified model and the initial model.

14. The electronic device according to claim 9 , wherein the obtaining the first target model by performing white model mapping rendering on the area of each material type on the mesh-simplified model, and obtaining the second target model by performing hyper-realistic rendering on the area of each mate

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Li Jie
• Liu Haojie
• Zhang Yan
• Shen Xuecen
• Chen Ruizhi
• Zhao Chen
• Teng Yuqiao
• Ding Errui
• Wu Tian
• Wang Haifeng

Classifications

• G06T15/04

  Texture mapping · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title

• G06T19/003

  Navigation within 3D models or images · CPC title

• G06T17/20

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title