Initializing a learned latent vector for neural-network projections of diverse images

Patent metadata

Field	Value
Publication number	US-11893717-B2
Application number	US-202117187080-A
Country	US
Kind code	B2
Filing date	Feb 26, 2021
Priority date	Feb 26, 2021
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

This disclosure describes one or more embodiments of systems, non-transitory computer-readable media, and methods that can learn or identify a learned-initialization-latent vector for an initialization digital image and reconstruct a target digital image using an image-generating-neural network based on a modified version of the learned-initialization-latent vector. For example, the disclosed systems learn a learned-initialization-latent vector from an initialization image utilizing a high number (e.g., thousands) of learning iterations on an image-generating-neural network (e.g., a GAN). Then, the disclosed systems can modify the learned-initialization-latent vector (of the initialization image) to generate modified or reconstructed versions of target images using the image-generating-neural network. For instance, the disclosed systems utilize the learned-initialization-latent vector as a starting point to learn a learned-latent vector for a target image that an image-generating-neural network converts into a high-fidelity reconstruction of the target image (with a reduced number of learning iterations).

First claim

What is claimed is:

1. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
identify a learned-initialization-latent vector learned from a first digital image utilizing an image-generating-neural network;
receive a modification request to modify a second digital image; and
modify the second digital image by:
modifying the learned-initialization-latent vector to generate a modified version of the learned-initialization-latent vector that, when processed by the image-generating-neural network, converts into a reconstructed version of the second digital image; and
generating a modified version of the second digital image utilizing the image-generating-neural network with the modified version of the learned-initialization-latent vector and the modification request.

2. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to identify at least one latent feature vector and multiple noise maps as part of the learned-initialization-latent vector.

3. The non-transitory computer-readable medium of claim 1 , wherein the image-generating-neural network comprises a generative-adversarial-neural network (GAN).

4. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to identify the learned-initialization-latent vector by identifying a latent vector that, when processed by the image-generating-neural network, converts into a reconstructed version of the first digital image.

5. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate a learned-latent vector by modifying the learned-initialization-latent vector into a particular modified version of the learned-initialization-latent vector that the image-generating-neural network converts into the reconstructed version of the second digital image instead of a reconstructed version of the first digital image.

6. The non-transitory computer-readable medium of claim 5 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
identify the learned-initialization-latent vector by iteratively modifying a latent vector for a first number of learning iterations to generate the reconstructed version of the first digital image utilizing the image-generating-neural network; and
generate, for the second digital image, the learned-latent vector by iteratively modifying the learned-initialization-latent vector for a second number of learning iterations to generate the reconstructed version of the second digital image utilizing the image-generating-neural network.

7. The non-transitory computer-readable medium of claim 5 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the learned-latent vector by:
iteratively modifying the learned-initialization-latent vector; and
generating, until satisfying a stopping condition, reconstructed digital images utilizing the image-generating-neural network based on modified versions of the learned-initialization-latent vector by:
generating, from the particular modified version of the learned-initialization-latent vector utilizing the image-generating-neural network, the reconstructed version of the second digital image that results in a threshold-satisfying loss based on a comparison between the reconstructed version of the second digital image and the second digital image;
iteratively modifying the learned-initialization-latent vector for a threshold number of learning iterations; or
iteratively modifying the learned-initialization-latent vector for a threshold period of time.

8. The non-transitory computer-readable medium of claim 7 , wherein the first digital image comprises an initialization digital image different from the second digital image.

9. A system comprising:
one or more memory devices comprising an image-generating-neural network and a first digital image; and
one or more processors configured to cause the system to:
identify a learned-initialization-latent vector learned from the first digital image utilizing the image-generating-neural network;
receive a modification request to modify a second digital image;
generate, for the second digital image, a learned-latent vector that, when processed by the image-generating-neural network, converts into a reconstructed version of the second digital image by:
iteratively modifying the learned-initialization-latent vector to generate modified versions of the learned-initialization-latent vector; and
generating reconstructed digital images utilizing the image-generating-neural network based on the modified versions of the learned-initialization-latent vector until reconstructing a version of the second digital image in place of the first digital image; and
generate a modified version of the second digital image utilizing the image-generating-neural network with the learned-latent vector and the modification request.

10. The system of claim 9 , wherein the one or more processors are configured to cause the system to identify a concatenation of at least one latent feature vector and multiple random noise maps as part of the learned-initialization-latent vector.

11. The system of claim 9 , wherein the image-generating-neural network comprises a generative-adversarial-neural network (GAN).

12. The system of claim 9 , wherein the one or more processors are configured to cause the system to project the first digital image into the learned-initialization-latent vector by iteratively modifying a latent vector into the learned-initialization-latent vector.

13. The system of claim 12 , wherein the one or more processors are configured to cause the system to:
iteratively modify the latent vector for a number of learning iterations to generate the reconstructed version of the first digital image utilizing the image-generating-neural network; and
generate the learned-latent vector by iteratively modifying the learned-initialization-latent vector for a lesser number of learning iterations to generate the reconstructed version of the second digital image utilizing the image-generating-neural network.

14. The system of claim 9 , wherein the one or more processors are configured to cause the system to:
determine at least one loss from a comparison between a particular reconstructed version of the second digital image and the second digital image; and
modify the learned-initialization-latent vector based on the at least one determined loss.

15. The system of claim 9 , wherein the one or more processors are configured to cause the system to iteratively modify the learned-initialization-latent vector for a threshold number of learning iterations.

16. The system of claim 9 , wherein the one or more processors are configured to cause the system to:
receive a request to modify the second digital image by combining the second digital image with a third digital image;
generate a modified version of the learned-latent vector by combining the learned-latent vector with an additional learned-latent vector corresponding to the third digital image; and
generate a modified version of the second digital image utilizing the image-generating-neural network based on the modified version of the learned-latent vector.

Assignees

• Adobe Inc

Inventors

• Tensmeyer Christopher
• Morariu Vlad
• Brodie Michael

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title