Adaptive self-adversarial negative sampling for graph neural network training

Patent metadata

Field	Value
Publication number	US-12505350-B2
Application number	US-202117161944-A
Country	US
Kind code	B2
Filing date	Jan 29, 2021
Priority date	Jan 29, 2021
Publication date	Dec 23, 2025
Grant date	Dec 23, 2025

Abstract

A graph neural network (GNN) training method, system, and computer program product in a graph, include generating, by the computing device, one or more one or more hypothetical edges between two or more nodes of a plurality of nodes of a graph neural network, testing, by the computing device, to determine whether the one or more generated hypothetical edges should be connected by using negative sampling, and permanently connecting, by the computing device, the one or more tested hypothetical edges if the negative sampling indicates the connectivity.

First claim

What is claimed is:

1 . A computer-implemented graph neural network (GNN) training method, comprising:
training a model in a first training iteration of the GNN; evaluating, using the trained model, a gradient caused by a negative sample of a plurality of negative samples; evaluating, using the trained model, a gradient caused by a positive sample, wherein the positive sample corresponds to the negative sample; classifying, using the trained model, the negative sample as a hard negative sample based on:
the negative sample causing a high gradient to the trained model for training in a next training iteration, and
the gradient caused by the negative sample being not higher than the gradient caused by the corresponding positive sample, wherein the corresponding positive sample is used during the classifying to offset a hardness of the negative sample to reduce false negative cases;
selecting the negative sample among the plurality of negative samples, via an Adaptive Self-Adversarial (ASA) negative sampling algorithm based on:
self-adversarial negative sampling,
the first training iteration of the GNN, and
the negative sample being classified as the hard negative sample, wherein a set of decay functions for the ASA automatically increases a threshold of the hardness for the next training iteration relative to the first training iteration; and
training the GNN in the next training iteration using the selected negative sample.

2 . The computer-implemented GNN training method of claim 1 , further comprising uniformly sampling a pool of candidates from possible negative links in the GNN, for reducing a selection space.

3 . The computer-implemented GNN training method of claim 2 , wherein a graph schema is defined to reduce a negative sample space which filters out a link in the GNN which is incompatible with the graph schema.

4 . The computer-implemented GNN training method of claim 1 , further comprising utilizing a training strategy for the ASA during an entire GNN training life-cycle.

5 . The computer-implemented GNN training method of claim 1 , wherein the computer-implemented GNN training method is embodied in a cloud-computing environment.

6 . A computer program product for graph neural network (GNN) training, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
training a model in a first training iteration of the GNN; evaluating, using the trained model, a gradient caused by a negative sample of a plurality of negative samples; evaluating, using the trained model, a gradient caused by a positive sample, wherein the positive sample corresponds to the negative sample; classifying, using the trained model, the negative sample as a hard negative sample based on:
the negative sample causing a high gradient to the trained model for training in a next training iteration, and
the gradient caused by the negative sample being not higher than the gradient caused by the corresponding positive sample, wherein the corresponding positive sample is used during the classifying to offset a hardness of the negative sample to reduce false negative cases;
selecting the negative sample among the plurality of negative samples, via an Adaptive Self-Adversarial (ASA) negative sampling algorithm based on:
self-adversarial negative sampling,
the first training iteration of the GNN, and
the negative sample being classified as the hard negative sample, wherein a set of decay functions for the ASA automatically increases a threshold of the hardness for the next training iteration relative to the first training iteration; and
training the GNN in the next training iteration using the selected negative sample.

7 . The computer program product of claim 6 , further comprising uniformly sampling a pool of candidates from possible negative links in the GNN, for reducing a selection space.

8 . The computer program product of claim 7 , wherein a graph schema is defined to reduce a negative sample space which filters out a link in the GNN which is incompatible with the graph schema.

9 . The computer program product of claim 6 , further comprising utilizing a training strategy for the ASA during an entire GNN training life-cycle.

10 . The computer program product of claim 6 , wherein the computer program product is embodied in a cloud-computing environment.

11 . A graph neural network (GNN) training system, comprising:
a processor; and a memory, the memory storing instructions to cause the processor to:
train a model in a first training iteration of the GNN;
evaluate, using the trained model, a gradient caused by a negative sample of a plurality of negative samples;
evaluate, using the trained model, a gradient caused by a positive sample, wherein the positive sample corresponds to the negative sample;
classify, using the trained model, the negative sample as a hard negative sample based on:
the negative sample that causes a high gradient to the trained model to train in a next training iteration, and
the gradient caused by the negative sample being not higher than the gradient caused by the corresponding positive sample, wherein the corresponding positive sample is used during the classifying to offset a hardness of the negative sample to reduce false negative cases;

select the negative sample among the plurality of negative samples, via an Adaptive Self-Adversarial (ASA) negative sampling algorithm based on:
self-adversarial negative sampling,
the first training iteration of the GNN, and
the negative sample being classified as the hard negative sample, wherein a set of decay functions for the ASA automatically increases a threshold of the hardness for the next training iteration relative to the first training iteration; and

train the GNN in the next training iteration using the selected negative sample.

12 . The GNN training system of claim 11 , wherein the memory further stores instructions to cause the processor to:
uniformly sample a pool of candidates from possible negative links in the GNN, for reducing a selection space.

13 . The GNN training system of claim 12 , wherein a graph schema is defined to reduce a negative sample space which filters out a link in the GNN which is incompatible with the graph schema.

14 . The GNN training system of claim 11 , wherein the memory further stores instructions to cause the processor to utilize a training strategy for the ASA during an entire GNN training life-cycle.

15 . The GNN training system of claim 11 , wherein the GNN training system is embodied in a cloud-computing environment.

16 . A computer-implemented graph neural network (GNN) training method, comprising:
training a model in a first training iteration of the GNN; evaluating, using the trained model, a gradient caused by a negative sample of a plurality of negative samples; evaluating, using the trained model, a gradient caused by a positive sample, wherein the positive sample corresponds to the negative sample; classifying, using the trained model, the negative sample as a hard negative sample based on:
the negative sample causing a high gradient to the trained model for training in a next training iteration, and
the gradient caused by the negative sample being not higher than the gradient caused by the corresponding positive sample, wherein the corresponding positive sample is used during the classifying to offset a hardness of the negative sample to reduce false negative cases;
se

Assignees

• IBM

Inventors

• Qin Xiao
• Sheikh Nasrullah
• Reinwald Berthold
• Wu Lingfei

Classifications

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title