Method and device for generative adversarial network training

Patent metadata

Field	Value
Publication number	US-2019130221-A1
Application number	US-201816179469-A
Country	US
Kind code	A1
Filing date	Nov 2, 2018
Priority date	Nov 2, 2017
Publication date	May 2, 2019
Grant date	—

Abstract

An electronic device for neural network training includes at least one processor and one or more memories configured to provide or train: a generative adversarial network (GAN) using a generator and a discriminator for: receiving a plurality of training cases; and training the generative adversarial network, based on the plurality of training cases, to classify the training cases; wherein the generator generates hard negative examples for the discriminator.

First claim

What is claimed is:

1 . An electronic device for neural network training comprising:
one or more processors; a non-transitory computer-readable medium storing one or more programs and data representative of a generative adversarial network (GAN) having a plurality of nodes and weights, the GAN including a generator and a discriminator, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving data representative of a plurality of training samples;
generating, by the generator, a plurality of hard negative samples based on the plurality of training samples; and
generating an output, by the discriminator, based on the plurality of hard negative samples from the generator.

2 . The electronic device of claim 1 , wherein at least one of the generator and the discriminator is a neutral network.

3 . The electronic device of claim 2 , wherein the generator is defined for an image or caption retrieval.

4 . The electronic device of claim 3 , wherein the generator for the image retrieval is based on:
λ p noise ( i )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images in the plurality of training cases.

5 . The electronic device of claim 3 , wherein the generator for the caption retrieval is based on:
λ p noise ( c )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images including correctly labelled images.

6 . The electronic device of claim 2 , wherein an output of the discriminator is used as a feedback to the generator.

7 . The electronic device of claim 3 , wherein the a score for an image-caption pair is used as a reward for the generator.

8 . The electronic device of claim 7 , wherein the GAN is configured to: receive data representative of an image, and generate an output to identify, from a plurality of text strings, a corresponding text string based on the data representative of the image.

9 . The electronic device of claim 7 , wherein the GAN is configured to: receive data representative of a text string, and generate an output to identify, from a plurality of images, a corresponding image based on the data representative of the text string.

10 . A computer-implemented method comprising:
receiving, by a generative adversarial network (GAN) having a plurality of nodes and weights, the GAN including a generator and a discriminator, data representative of a plurality of training samples; generating, by the generator, a plurality of hard negative samples based on the plurality of training samples; and generating an output, by the discriminator, based on the plurality of hard negative samples from the generator.

11 . The method of claim 10 , wherein at least one of the generator and the discriminator is a neutral network.

12 . The method of claim 11 , wherein the generator for the image retrieval is based on:
λ p noise ( i )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images in the plurality of training cases.

13 . The method of claim 11 , wherein the generator for the caption retrieval is based on:
λ p noise ( c )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images including correctly labelled images.

14 . The method of claim 10 , wherein an output of the discriminator is used as a feedback to the generator.

15 . The method of claim 11 , wherein the a score for an image-caption pair is used as a reward for the generator.

16 . The method of claim 15 , wherein the GAN is configured to: receive data representative of an image, and generate an output to identify, from a plurality of text strings, a corresponding text string based on the data representative of the image.

17 . The method of claim 15 , wherein the GAN is configured to: receive data representative of a text string, and generate an output to identify, from a plurality of images, a corresponding image based on the data representative of the text string.

18 . An electronic device comprising:
one or more processors; and a memory storing one or more program configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving a text string or an image;
processing the text string or the image using a generative adversarial network including a generator and a discriminator; and
choosing an matched image based on the processed text string from a plurality of images, or choosing a matched text string based on the processed image from a plurality of text strings;
wherein during a training phase, the generator is configured to generate a plurality of hard negatives samples for the discriminator.

19 . The method of claim 18 , wherein the generator is based on:
λ p noise ( i )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images in a plurality of training cases.

20 . The method of claim 18 , wherein the generator for is based on:
λ p noise ( c )+(1−λ) g θ ( i|c )
wherein p noise is a fixed noise distribution, g θ is a conditional distribution with learnable parameters θ and λ is a hyperparameter, and g θ (i|c) defines a categorical distribution over all possible images from a set of images including correctly labelled images.

Assignees

• Royal Bank Of Canada

Inventors

• Bose Avishek
• Cao Yanshuai

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06F18/24

  Classification techniques · CPC title