Meta cooperative training paradigms

Patent metadata

Field	Value
Publication number	US-12293284-B2
Application number	US-202017136054-A
Country	US
Kind code	B2
Filing date	Dec 29, 2020
Priority date	Feb 5, 2020
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

Generative adversarial models have several benefits; however, due to mode collapse, these generators face a quality-diversity trade-off (i.e., the generator models sacrifice generation diversity for increased generation quality). Presented herein are embodiments that improve the performance of adversarial content generation by decelerating mode collapse. In one or more embodiments, a cooperative training paradigm is employed where a second model is cooperatively trained with the generator and helps efficiently shape the data distribution of the generator against mode collapse. Moreover, embodiments of a meta learning mechanism may be used, where the cooperative update to the generator serves as a high-level meta task and which helps ensures the generator parameters after the adversarial update stay resistant against mode collapse. In experiments, tested employments demonstrated efficient slowdown of mode collapse for the adversarial text generators. Overall, embodiments outperformed the baseline approaches with significant margins in terms of both generation quality and diversity.

First claim

What is claimed is:

1. A computer-implemented method for training a generator comprising:
responsive to a stop condition having not been reached, performing steps comprising:
sampling a set of data from a training data;
using a generator model, which comprises a set of generator parameter values, to generate a set of generated data;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using a set of data sampled from the training data as inputs:
into a second neural network model, which comprises a second neural network model set of parameter values, to obtain one or more output distributions from the second neural network model; and
into the generator model comprising the set of intermediate generator parameter values to obtain one or more output distributions from the generator model;

determining a meta gradient for a cooperate training loss that comprises comparing one or more output distributions from the second neural network model with one or more corresponding output distributions from the generator model;
updating a set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient;
updating a set of discriminator parameter values for a discriminator model using an adversarial loss for the discriminator model; and
updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss for the second neural network model; and

responsive to the stop condition having been reached, outputting the generator model, which comprises a final updated set of generator parameter values.

2. The computer-implemented method of claim 1 further comprising as initial steps:
initializing at least the set of generator parameter values of the generator model and the set of discriminator parameter values of the discriminator model; and
pretraining the generator model using training data, the generator model, and the discriminator model.

3. The computer-implemented method of claim 2 wherein the second neural network model and the generator model share a same neural network structure and the method further comprises:
using at least some of the set of generator parameter values from the pretrained generator model as parameter values for the second neural network model.

4. The computer-implemented method of claim 1 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
using a maximum likelihood estimation (MLE) loss function.

5. The computer-implemented method of claim 4 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between:
one or more outputs from the second neural network model using a set of data sampled from the training data; and
one or more outputs from the second neural network model using a mixture of data sampled from the training data and data sampled from data that were generated by the generator model.

6. The computer-implemented method of claim 5 wherein the mixture comprises an equal number or approximately equal number of data from the training data and data points which were generated by the generator model.

7. The computer-implemented method of claim 1 wherein the adversarial loss for the discriminator model and the adversarial loss for the generator model are obtained by using a min-max loss function.

8. A system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
responsive to a stop condition having not been reached, performing steps comprising:
sampling a set of data from a training data having a first distribution;
using a generator model, which comprises a set of generator parameter values, to generate a set of generated data;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using a set of data sampled from the training data as inputs:
into a second neural network model, which comprises a second neural network model set of parameter values, to obtain one or more output distributions from the second neural network model; and
into the generator model comprising the set of intermediate generator parameter values to obtain one or more output distributions from the generator model;

determining a meta gradient for a cooperate training loss that comprises comparing one or more output distributions from the second neural network model with one or more corresponding output distributions from the generator model;
updating a set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient;
updating a set of discriminator parameter values for a discriminator model using an adversarial loss for the discriminator model; and
updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss for the second neural network model; and

responsive to the stop condition having been reached, outputting the generator model, which comprises a final updated set of generator parameter values.

9. The system of claim 8 wherein the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
initializing at least the set of generator parameter values of the generator model and the set of discriminator parameter values of the discriminator model; and
pretraining the generator model using training data, the generator model, and the discriminator model.

10. The system of claim 9 wherein the second neural network model and the generator model share a same neural network structure and the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
using at least some of the set of generator parameter values from the pretrained generator model as parameter values for the second neural network model.

11. The system of claim 8 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
using a maximum likelihood estimation (MLE) loss function.

12. The system of claim 11 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between:
one or more outputs from the second neural network model using a set of data sampled from the training data; and
one or more outputs from the second neural network model using a mixture of data sampled from the training data and data sampled from data that were generated by the generator model.

13. The system of claim 12 wherei

Assignees

• Baidu Usa Llc

Inventors

• Li Dingcheng
• Yin Haiyan
• Li Xu
• Li Ping

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title