Agilegan-based stylization method to enlarge a style region

Patent metadata

Field	Value
Publication number	US-12190481-B2
Application number	US-202217807527-A
Country	US
Kind code	B2
Filing date	Jun 17, 2022
Priority date	Jun 17, 2022
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

Methods and systems for enlarging a stylized region of an image are disclosed that include receiving an input image, generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image, normalizing the input image, generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image, blending the first stylized image and the second stylized image to obtain a third stylized image, and providing the third stylized image as an output.

First claim

What is claimed is:

1. A method for enlarging a style region of an image, the method comprising:
receiving an input image;
generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image;
normalizing the input image;
generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image;
extracting a first face parse mask from the first stylized image, wherein the first face parse mask includes one or more stylized features of the first stylized image;
extracting a second face parse mask from the second stylized image, wherein the second face parse mask includes one or more stylized features of the second stylized image;
blending the first stylized image and the second stylized image to obtain a third stylized image, wherein the blending of the first stylized image and the second stylized image is based on the first and second face parse masks; and
providing the third stylized image as an output.

2. The method of claim 1 , wherein the blending of the first stylized image and the second stylized image comprises gaussian blending.

3. The method of claim 2 , wherein the third stylized image comprises a first set of pixels corresponding to hair and necks regions of the first stylized image, and a second set of pixels corresponding to the face region of the second stylized image.

4. The method of claim 1 , wherein the normalizing comprises one or more of cropping or scaling the input image.

5. The method of claim 1 , wherein the first GAN generator is different than the second GAN generator.

6. The method of claim 1 , wherein the first GAN generator and the second GAN generator are AgileGAN generators.

7. The method of claim 1 , further comprising:
receiving a plurality of exemplar stylistic images; and
training the first and second GAN generators using transfer learning based on the received plurality of exemplar stylistic images.

8. A system, comprising:
at least one processor;
memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations including:
receiving an input image;
generating, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image;
normalizing the input image;
generating, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image;
extracting a first face parse mask from the first stylized image, wherein the first face parse mask includes one or more stylized features of the first stylized image;
extracting a second face parse mask from the second stylized image, wherein the second face parse mask includes one or more stylized features of the second stylized image;
blending the first stylized image and the second stylized image to obtain a third stylized image, wherein the blending of the first stylized image and the second stylized image is based on the first and second face parse masks; and
providing the third stylized image as an output.

9. The system of claim 8 , wherein the blending of the first stylized image and the second stylized image comprises gaussian blending.

10. The system of claim 8 , wherein the third stylized image comprises a first set of pixels corresponding to hair and necks regions of the first stylized image, and a second set of pixels corresponding to the face region of the second stylized image.

11. The system of claim 8 , wherein the normalizing comprises one or more of cropping or scaling the input image.

12. The system of claim 8 , wherein the first GAN generator is different than the second GAN generator.

13. The system of claim 8 , wherein the first GAN generator and the second GAN generator are AgileGAN generators.

14. The system of claim 8 , wherein the set of operations further include:
receiving a plurality of exemplar stylistic images; and
training the first and second GAN generators using transfer learning based on the received plurality of exemplar stylistic images.

15. A non-transitory computer-readable storage medium comprising instructions being executable by one or more processors to cause the one or more processors to:
receive an input image;
generate, using a first generative adversarial network (GAN) generator, a first stylized image, based on the input image;
normalize the input image;
generate, using a second generative adversarial network (GAN) generator, a second stylized image, based on the normalized input image;
extract a first face parse mask from the first stylized image, wherein the first face parse mask includes one or more stylized features of the first stylized image;
extract a second face parse mask from the second stylized image, wherein the second face parse mask includes one or more stylized features of the second stylized image;
blend the first stylized image and the second stylized image to obtain a third stylized image, wherein the blending of the first stylized image and the second stylized image is based on the first and second face parse masks; and
provide the third stylized image as an output.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the instructions further cause the one or more processors to:
receive a plurality of exemplar stylistic images; and
train the first and second GAN generators using transfer learning based on the received plurality of exemplar stylistic images.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the third stylized image comprises a first set of pixels corresponding to hair and necks regions of the first stylized image, and a second set of pixels corresponding to the face region of the second stylized image.

Assignees

• Lemon Inc

Inventors

• Song Guoxian
• Liu Jing
• Zeng Weihong
• Sun Jingna
• Wang Xu
• Luo Linjie

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20132

  Image cropping · CPC title

• G06T2207/30201

  Face · CPC title