Method for generating human-computer interactive abstract image

Patent metadata

Field	Value
Publication number	US-11734389-B2
Application number	US-202117239696-A
Country	US
Kind code	B2
Filing date	Apr 26, 2021
Priority date	Dec 7, 2020
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

A method for generating a human-computer interactive abstract image includes: S 1 : obtaining and preprocessing the original abstract images used as a training dataset B to obtain edge shape feature maps used as a training dataset A; S 2 : using the training dataset A and the training dataset B as cycle generative objects of a Cycle-GAN model, and training the Cycle-GAN model to capture a mapping relationship between the edge shape feature maps and the original abstract images; S 3 : obtaining a line shape image drawn by a user; and S 4 : according to the mapping relationship, intercepting a generative part in the Cycle-GAN model that the dataset B is generated from the dataset A, discarding a cycle generative part and a discrimination part in the Cycle-GAN model, and generating a complete abstract image based on the line shape image to generate the human-computer interactive abstract image.

First claim

What is claimed is:

1. A method for generating a human-computer interactive abstract image, comprising the steps of:
S 1 : obtaining original abstract images, and preprocessing the original abstract images to obtain edge shape feature maps in one-to-one correspondence with the original abstract images; wherein the edge shape feature maps are used as a training dataset A, and the original abstract images are used as a training dataset B;
S 2 : using the training dataset A and the training dataset B as cycle generative objects of a Cycle-generative adversarial network (GAN) model, and training the Cycle-GAN model to capture a mapping relationship between the edge shape feature maps and the original abstract images;
S 3 : obtaining a line shape image drawn by a user; and
S 4 : according to the mapping relationship, intercepting a generative part in the Cycle-GAN model that the training dataset B is generated from the training dataset A, discarding a cycle generative part and a discrimination part in the Cycle-GAN model, and generating a complete abstract image based on the line shape image to generate the human-computer interactive abstract image.

2. The method according to claim 1 , wherein step S 1 comprises:
S 101 : obtaining the original abstract images, and using the original abstract images to construct the training dataset B;
S 102 : performing a binarization processing on the original abstract images in the training dataset B to obtain binarized images, and extracting color edge information in the binarized images to obtain the edge shape feature maps in one-to-one correspondence with the original abstract images; and
S 103 : calculating lengths of edge lines of the edge shape feature maps, and discarding edge lines with a length being greater than 150 pixels to obtain the training dataset A.

3. The method according to claim 1 , wherein
the Cycle-GAN model in step S 2 comprises a first generator G, a second generator F, a first discriminator D G and a second discriminator D F ; wherein the first generator G and the second generator F are identical structurally, and the first discriminator D G and the second discriminator D F are identical structurally;
the first generator G is configured to capture the mapping relationship between the edge shape feature maps and the original abstract images;
the second generator F is configured to capture an inverse mapping relationship between the edge shape feature maps and the original abstract images;
the first discriminator D G is configured to discriminate a generative quality of the first generator G; and
the second discriminator D F is configured to discriminate a generative quality of the second generator F.

4. The method according to claim 3 , wherein
each of the first discriminator D G and the second discriminator D F comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer and a fifth convolutional layer, wherein the first convolutional layer, the second convolutional layer, the third convolutional layer, the fourth convolutional layer and the fifth convolutional layer are successively connected; each of the first convolutional layer, the second convolutional layer, the third convolutional layer and the fourth convolutional layer is provided with a first normalization operation and a rectified linear unit (ReLU) activation function; the fifth convolutional layer is provided with a Sigmoid function; and
each of the first generator G and the second generator F comprises an encoding module, a residual module and a decoding module, wherein the encoding module, the residual module and the decoding module are successively connected.

5. The method according to claim 4 , wherein
a number of convolutional kernels of the first convolutional layer is 64, a size of the convolutional kernels of the first convolutional layer is 4×4, and a stride of the first convolutional layer is 2;
a number of convolutional kernels of the second convolutional layer is 128, a size of the convolutional kernels of the second convolutional layer is 4×4, and a stride of the second convolutional layer is 2;
a number of convolutional kernels of the third convolutional layer is 256, a size of the convolutional kernels of the third convolutional layer is 4×4, and a stride of the third convolutional layer is 2;
a number of convolutional kernels of the fourth convolutional layer is 512, a size of the convolutional kernels of the fourth convolutional layer is 4×4, and a stride of the fourth convolutional layer is 2; and
a number of convolutional kernel of the fifth convolutional layer is 1, a size of the convolutional kernel of the fifth convolutional layer is 4×4, and a stride of the fifth convolutional layer is 1.

6. The method according to claim 5 , wherein
the encoding module comprises a sixth convolutional layer, a seventh convolutional layer and an eighth convolutional layer, wherein the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer are successively connected; each of the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer is provided with a second normalization operation and the ReLU activation function;
the residual module comprises a first residual layer, a second residual layer, a third residual layer, a fourth residual layer, a fifth residual layer and a sixth residual layer, wherein the first residual layer, the second residual layer, the third residual layer, the fourth residual layer, the fifth residual layer and the sixth residual layer are successively connected; each of the first residual layer, the second residual layer, the third residual layer, the fourth residual layer, the fifth residual layer and the sixth residual layer is provided with the second normalization operation and the ReLU activation function;
the decoding module comprises a first decoding layer, a second decoding layer and a third decoding layer, wherein the first decoding layer, the second decoding layer and the third decoding layer are successively connected; each of the first decoding layer and the second decoding layer is provided with the second normalization operation and the ReLU activation function; the third decoding layer is provided with a Tanh function; and
the eighth convolutional layer is connected to the first residual layer, and the sixth residual layer is connected to the first decoding layer.

7. The method according to claim 6 , wherein
a number of convolutional kernels of the sixth convolutional layer is 32, a size of the convolutional kernels of the sixth convolutional layer is 7×7, and a stride of the sixth convolutional layer is 1;
a number of convolutional kernels of the seventh convolutional layer is 64, a size of the convolutional kernels of the seventh convolutional layer is 3×3, and a stride of the seventh convolutional layer is 2;
a number of convolutional kernels of the eighth convolutional layer is 128, a size of the convolutional kernels of the eighth convolutional layer is 3×3, and a stride of the eighth convolutional layer is 2;
each of the first residual layer, the second residual layer, the third residual layer, the fourth residual layer, the fifth residual layer and the sixth residual layer comprises two convolutional layers; a number of convolutional kernels of each of the two convolutional layers is 128, a size of the convolutional kernels of each of the two convolutional layers is 3×3, and a stride of each of the two convolutional layers is 1;
a number of convolutional kernels of the first decoding layer is 64, a size of the convolutional kernels of the first decoding layer is 3×3, and a stride of the first decoding layer is 2;
a number of convolutio

Assignees

• Univ Sichuan

Inventors

• Lv Jiancheng
• Huang Youcheng
• Li Mao
• Tang Chenwei
• Liu Quanhui

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06F18/2132 Primary

  based on discrimination criteria, e.g. discriminant analysis · CPC title