Method and system for generating 3d image of character

Patent metadata

Field	Value
Publication number	US-2020184711-A1
Application number	US-201916691160-A
Country	US
Kind code	A1
Filing date	Nov 21, 2019
Priority date	Dec 7, 2018
Publication date	Jun 11, 2020
Grant date	—

Abstract

A method and a system for generating a 3D image of a character through steps of: receiving an identifier of a template and a two-dimensional (2D) image of the character generated based on the template corresponding to a type of the character; acquiring template information of the template from a template library by using the identifier, and extracting an effective parameter for machine learning to be performed to generate the 3D image based on the template information and the 2D image; and generating the 3D image of the character by performing the machine learning based on the effective parameter and the template information are provided.

First claim

What is claimed is:

1 . A method for generating a three-dimensional (3D) image of a character, the method comprising:
receiving an identifier of a template and a two-dimensional (2D) image of the character generated based on the template corresponding to a type of the character; acquiring template information of the template from a template library by using the identifier, and extracting an effective parameter for machine learning to be performed to generate the 3D image based on the template information and the 2D image; and generating the 3D image of the character by performing the machine learning based on the effective parameter and the template information.

2 . The method of claim 1 , wherein the template information includes a gender of the character and an age of the character, and the 2D image represents a character of the age having the gender.

3 . The method of claim 2 , wherein the template information further includes style information of the character, and
wherein generating the 3D image of the character by performing the machine learning based on the effective parameter and the template information includes performing the machine learning by using an inference engine that transforms a 3D image according to the style information.

4 . The method of claim 1 , wherein the template includes the template information and a silhouette, and the 2D image is an image of the character produced based on the silhouette.

5 . The method of claim 4 , wherein the effective parameter includes a plurality of template images mapped to each body part of the silhouette.

6 . The method of claim 5 , further comprising
determining a correlation between body parts of the character based on the 2D image and a three-dimensional template of the template, wherein the generating the 3D image of the character by performing the machine learning based on the effective parameter and the template information includes performing the machine learning to synthesize the plurality of template images by using the correlation.

7 . A system for generating a three-dimensional (3D) image of a character, the system comprising:
a template library configured to store a plurality of templates representing the type of the character; a pre-processing unit configured to extract an effective parameter for machine learning to be performed to generate the 3D image based on template information of a first template of the plurality of templates and a two-dimensional (2D) image of the character which is generated based on the first template; and a modeler configured to generate the 3D image of the character by performing the machine learning based on the effective parameter.

8 . The system of claim 7 , wherein the template information includes a gender of the character and an age of the character, and the 2D image represents a character of the age having the gender.

9 . The system of claim 8 , wherein
the template information further includes style information of the character, and wherein the modeler further configured to perform the machine learning by using an inference engine that transforms the 3D image according to the style information.

10 . The system of claim 7 , wherein the first template includes the template information and a silhouette, wherein the 2D image is an image of the character produced based on the silhouette.

11 . The system of claim 10 , wherein the effective parameter comprises a plurality of template images mapped to each body part of the silhouette.

12 . The system of claim 5 , wherein the pre-processing unit further configured to determine a correlation between body parts of the character based on the a 3D template of the template and the 2D image,
wherein the modeler further configured to perform the machine learning to synthesize the plurality of template images by using the correlation.

13 . An apparatus for pre-processing image information of a character, the apparatus comprising:
a processor, a memory, and a transmission/reception interface, wherein the processor executes a program stored in the memory to perform: receiving an identifier of a template and a two-dimensional (2D) image of the character generated on the basis of the template corresponding to a type of the character through the transmission/reception interface; acquiring template information of the template by using the identifier, and extract an effective parameter for machine learning to be performed to generate a three-dimensional (3D) image by using the template information and the 2D image; and structuralizing the template information and the effective parameter into a data structure and transmitting the data structure to the modeler performing the machine learning through the transmission/reception interface.

14 . The apparatus of claim 13 , wherein
the processor executes the program to further perform determining a correlation between body parts of the character based on a 3D template of the template and the 2D image, and the processor, when structuralizing the template information and the effective parameter into a data structure and transmitting the data structure to the modeler performing the machine learning through the transmission/reception interface, performs structuralizing the correlation into the data structure along with the template information and the effective parameter.

15 . The apparatus of claim 13 , wherein the template information includes style information of the character, the data structure includes instruction information regarding an inference neural network of the modeler, and the instruction information indicates an inference neural network corresponding to the body part to the modeler.

16 . The apparatus of claim 13 , wherein the template information includes style information of the character, the data structure includes instruction information regarding an inference neural network of the modeler, and the instruction information indicates an inference neural network corresponding to the style information to the modeler.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Choi Jin Sung
• Lee Seung Wook
• Kim Tae-Joon
• Yoon Seung Uk
• Lim Seong Jae
• Hwang Bon Woo
• Park Chang Joon

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title