Progressive modification of generative adversarial neural networks

Patent metadata

Field	Value
Publication number	US-2019130278-A1
Application number	US-201816156994-A
Country	US
Kind code	A1
Filing date	Oct 10, 2018
Priority date	Oct 26, 2017
Publication date	May 2, 2019
Grant date	—

Abstract

A generative adversarial neural network (GAN) learns a particular task by being shown many examples. In one scenario, a GAN may be trained to generate new images including specific objects, such as human faces, bicycles, etc. Rather than training a complex GAN having a predetermined topology of features and interconnections between the features to learn the task, the topology of the GAN is modified as the GAN is trained for the task. The topology of the GAN may be simple in the beginning and become more complex as the GAN learns during the training, eventually evolving to match the predetermined topology of the complex GAN. In the beginning the GAN learns large-scale details for the task (bicycles have two wheels) and later, as the GAN becomes more complex, learns smaller details (the wheels have spokes).

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
training, for a first time duration, a generative adversarial network (GAN) comprising a generator neural network coupled to a discriminator neural network, wherein a topology of the GAN comprises features and interconnections between the features within the generator neural network and within the discriminator neural network; modifying the topology of the GAN to produce a modified GAN; and training the modified GAN for a second time duration.

2 . The computer-implemented method of claim 1 , wherein modifying the topology changes a processing capacity of the generator neural network.

3 . The computer-implemented method of claim 1 , wherein the topology is modified by adding at least one layer in the generator neural network.

4 . The computer-implemented method of claim 1 , wherein the topology is modified by adding at least one layer in the discriminator neural network.

5 . The computer-implemented method of claim 1 , wherein the topology is modified by removing at least one layer in the generator neural network.

6 . The computer-implemented method of claim 1 , wherein training data includes example output data, and, further comprising, during the training of the modified GAN:
processing input data by the generator neural network to produce output data; modifying the example output data to produce modified training data; and processing the modified training data and the output data by the discriminator neural network to produce updated parameters for the GAN.

7 . The computer-implemented method of claim 6 , wherein the modified training data for the first time duration is different compared with the modified training data for the second time duration.

8 . The computer-implemented method of claim 6 , wherein the modified training data for the first time duration is modified according to a first function and the modified training data for the second time duration is modified according to a second function that is different than the first function.

9 . The computer-implemented method of claim 6 , wherein the training data further comprises additional input data and the additional input data are paired with the example output data.

10 . The computer-implemented method of claim 6 , wherein modifying the training data comprises increasing or decreasing a density of the example output data.

11 . The computer-implemented method of claim 6 , wherein the training data is image data and modifying the training data comprises decreasing a pixel resolution of the training data.

12 . The computer-implemented method of claim 11 , wherein an amount by which the pixel resolution of the training data is decreased for the first time duration is greater compared with the second time duration.

13 . The computer-implemented method of claim 1 , further comprising, during the second time duration, smoothly modifying the topology.

14 . The computer-implemented method of claim 13 , wherein first intermediate values generated by the GAN using the topology are interpolated with second intermediate values generated by the GAN using the modified topology.

15 . The computer-implemented method of claim 1 , wherein the GAN processes three-dimensional image data.

16 . The computer-implemented method of claim 1 , wherein the GAN processes audio data.

17 . A system, comprising:
a generative adversarial network (GAN) comprising a generator neural network coupled to a discriminator neural network, wherein the GAN is trained for a first time duration and a topology of the GAN comprises features and interconnections between the features within the generator neural network and within the discriminator neural network; the topology of the GAN is modified to produce a modified GAN; and the modified GAN is trained for a second time duration.

18 . The system of claim 17 , wherein modifying the topology changes a processing capacity of the generator neural network.

19 . The system of claim 17 , wherein the topology is modified by adding at least one layer in the generator neural network.

20 . A non-transitory computer-readable media storing computer instructions for training a generative adversarial network (GAN) comprising a generator neural network coupled to a discriminator neural network that, when executed by one or more processors, cause the one or more processors to perform the steps of:
training the GAN for a first time duration, wherein a topology of the GAN comprises features and interconnections between the features within the generator neural network and within the discriminator neural network; modifying the topology of the GAN to produce a modified GAN; and training the modified GAN for a second time duration.

21 . A computer-implemented method, for training a generative adversarial network (GAN) which includes a generator neural network coupled to a discriminator neural network, the method comprising:
receiving example output data at an input of the GAN; the generator neural network processing input data to produce generator output data; the discriminator neural network comparing the example output data to the generator output data, and
if the generator output data sufficiently matches the example output data according to a criterion, outputting a first training stimulus, and
if the generator output data does not sufficiently match the example output data according to the criterion, outputting a second training stimulus;
in response to the discriminator neural network outputting the second training stimulus, modifying at least one of layers, features, and interconnections within at least one of the generator neural network and the discriminator neural network, whereby the GAN is modified.

Assignees

• Nvidia Corp

Inventors

• Karras Tero Tapani
• Aila Timo Oskari
• Laine Samuli Matias
• Lehtinen Jaakko T

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title