Method for training image generation model, method for generating images, and devices thereof

Patent metadata

Field	Value
Publication number	US-12475689-B2
Application number	US-202218263854-A
Country	US
Kind code	B2
Filing date	Jan 28, 2022
Priority date	Feb 2, 2021
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

Provided is a method for training an image generation model, including: acquiring a first transformation model by training; acquiring a reconstruction model by training based on the first transformation model; acquiring a second transformation model by training; generating a grafted transformation model by grafting the first transformation model with the second transformation model; and generating the image generation model based on the reconstruction model and the grafted transformation model. The first transformation model is configured to generate a first training image according to a first noise sample. The first training image is an image of a first style. The reconstruction model is configured to map an original image sample to a latent variable corresponding to the original image sample. The second transformation model is configured to generate a second training image according to a second noise sample. The second training image is an image of a second style.

First claim

What is claimed is:

1 . A method for training an image generation model, comprising:
acquiring a first transformation model by training, wherein the first transformation model is configured to generate a first training image according to a first noise sample, and the first training image is an image of a first style; acquiring a reconstruction model by training based on the first transformation model, wherein the reconstruction model is configured to map an original image sample to a latent variable corresponding to the original image sample; acquiring a second transformation model by training, wherein the second transformation model is configured to generate a second training image according to a second noise sample, and the second training image is an image of a second style; generating a grafted transformation model by grafting the first transformation model with the second transformation model; and generating the image generation model based on the reconstruction model and the grafted transformation model, wherein the image generation model is configured to transform an image to be transformed of the first style into a target image of the second style.

2 . The method according to claim 1 , wherein
the first transformation model comprises a first mapping network and a first synthesis network; and acquiring the first transformation model by training comprises:
acquiring a first training sample set, wherein the first training sample set comprises a plurality of first noise samples;
acquiring latent variables respectively corresponding to the plurality of first noise samples by inputting the plurality of first noise samples into the first mapping network;
acquiring first training images respectively corresponding to the plurality of first noise samples by inputting the latent variables respectively corresponding to the plurality of first noise samples into the first synthesis network; and
adjusting a weight parameter of the first transformation model based on the first training images respectively corresponding to the plurality of first noise samples.

3 . The method according to claim 2 , wherein
the first transformation model comprises a first discrimination network; and adjusting the weight parameter of the first transformation model based on the first training images respectively corresponding to the plurality of first noise samples comprises:
acquiring first discrimination losses respectively corresponding to the plurality of first noise samples by inputting a plurality of first training images respectively corresponding to the plurality of first noise samples into the first discrimination network; and
adjusting the weight parameter of the first transformation model based on the first discrimination losses respectively corresponding to the plurality of first noise samples.

4 . The method according to claim 1 , wherein acquiring the reconstruction model based on the first transformation model comprises:
acquiring a second training sample set, wherein the second training sample set comprises a plurality of original image samples; generating latent variables respectively corresponding to the plurality of original image samples by inputting the plurality of original image samples into the reconstruction model; generating reconstructed images respectively corresponding to the plurality of original image samples by inputting the latent variables respectively corresponding to the plurality of original image samples into the first transformation model, wherein the plurality of original image samples and the reconstructed images respectively corresponding to the plurality of original image samples are images of the first style; determining losses of the reconstruction model respectively corresponding to the plurality of original image samples based on the plurality of original image samples and the reconstructed images respectively corresponding to the plurality of original image samples; and adjusting a weight parameter of the reconstruction model based on the losses of the reconstruction model respectively corresponding to the plurality of original image samples.

5 . The method according to claim 4 , wherein
the first transformation model comprises a first discrimination network; and determining the losses of the reconstruction model respectively corresponding to the plurality of original image samples based on the plurality of original image samples and the reconstructed images respectively corresponding to the plurality of original image samples comprises:
determining a first sub-loss based on an output result acquired by inputting each of the reconstructed images respectively corresponding to the plurality of original image samples into the first discrimination network, wherein the first sub-loss indicates a first characterization of the reconstructed image;
determining a second sub-loss based on an output result acquired by inputting each of the plurality of original image samples and each of the reconstructed images respectively corresponding to the plurality of original image samples into a perceptual network, wherein the second sub-loss indicates a first degree of conformity of the original image sample to the reconstructed image corresponding to the original image sample with respect to a target feature;
determining a third sub-loss based on an output result acquired by inputting each of the plurality of original image samples and each of the reconstructed images respectively corresponding to the plurality of original image samples into a regression function, wherein the third sub-loss indicates a second degree of conformity of the original image sample to the reconstructed image corresponding to the original image sample with respect to the target feature; and
determining the losses of the reconstruction model based on the first sub-loss, the second sub-loss, and the third sub-loss.

6 . The method according to claim 1 , wherein during a training process, an initial weight parameter of the second transformation model is a weight parameter of the first transformation model.

7 . The method according to claim 1 , wherein generating the grafted transformation model by grafting the first transformation model with the second transformation model comprises:
generating the grafted transformation model by grafting n layers of weight network in a plurality of layers of weight network of the first transformation model with m layers of weight network in a plurality of layers of weight network of the second transformation model; wherein layers of the n layers of weight network and the m layers of weight network are different, n is a positive integer, and m is a positive integer; or generating the grafted transformation model by performing a summing or averaging or difference operation on weight parameters of the plurality of layers of weight network of the first transformation model and corresponding weight parameters of the plurality of layers of weight network of the second transformation model.

8 . The method according to claim 1 , wherein generating the image generation model based on the reconstruction model and the grafted transformation model comprises:
acquiring a combined transformation model by combining the reconstruction model and the grafted transformation model; acquiring a fourth training sample set, wherein the fourth training sample set comprises at least one original image sample and an image of the second style corresponding to the at least one original image sample; and generating the image generation model by fine-tuning the combined transformation model using the fourth training sample set.

9 . A computer device for training an image generation model, comprising: a pro

Assignees

• Bigo Tech Pte Ltd

Inventors

• Li An
• Li Yule
• Xiang Wei

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title