Iterative deep graph learning for graph neural networks

Patent metadata

Field	Value
Publication number	US-12475694-B2
Application number	US-202016883189-A
Country	US
Kind code	B2
Filing date	May 26, 2020
Priority date	May 26, 2020
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

An initial noisy graph topology is obtained and an initial adjacency matrix is generated by a similarity learning component using similarity learning and a similarity metric function. An updated adjacency matrix with node embeddings is produced from the initial adjacency matrix using a graph neural network (GNN). The node embeddings are fed back to revise the similarity learning component. The generating, producing, and feeding back operations are repeated for a plurality of iterations.

First claim

What is claimed is:

1 . A method for jointly learning a graph structure and graph embeddings, the method comprising:
obtaining an initial noisy graph topology; generating, using a similarity learning component, an initial adjacency matrix using similarity learning and a similarity metric function; producing, from said initial adjacency matrix, using a graph neural network (GNN), an updated adjacency matrix with node embeddings; feeding back the node embeddings to revise the similarity learning component; repeating the generating, producing, and feeding back operations for a plurality of iterations; updating weights of the graph neural network only after a convergence of the updated adjacency matrices generated by the plurality of iterations; performing natural language processing on input from a human subject using a selected iteration of the updated adjacency matrix; and reconfiguring at least one network-based asset in accordance with a result of the natural language processing.

2 . The method of claim 1 , further comprising controlling, by a graph regularization component, a smoothness, a connectivity and a sparsity of a corresponding graph structure of the updated adjacency matrix using an adapted graph regularization technique.

3 . The method of claim 1 , wherein the similarity metric function comprises a weighted cosine similarity metric applied to obtain the updated adjacency matrix for all pairs of nodes of a corresponding graph structure.

4 . The method of claim 3 , wherein the weighted cosine similarity metric is defined using m weight vectors w, each weight vector w representing one perspective, to compute m independent similarity matrices s, an average of the m independent similarity matrices s, and a final similarity s based on:

s

i
⁢
j

k

=

cos
⁡
(

w
k

⊙

v
i

,

w
k

⊙

v
j

)

,

s

i
⁢
j

=

1
m

⁢

∑

k
=
1

m

s

i
⁢
j

k

where s ij k computes a cosine similarity between input vectors v i and v j for a k-th perspective, where each perspective considers one part of semantics captured in the input vectors v i and v j .

5 . The method of claim 4 , wherein the adjacency matrix is designated as A (t) and is extracted from S by considering elements in S which are smaller than a non-negative threshold ε in an ε-neighborhood for each node, where:

A
ij

=

{

s

i
⁢
j

s

i
⁢
j

>
ε

0

otherwise

.

6 . The method of claim 3 , wherein the weighted cosine similarity metric is defined as:
s ij =cos( w⊙v i ,w⊙v j ) where ⊙ denotes a Hadamard product and w comprises a learnable weight vector which has a same dimension as input vectors v i and v j .

7 . The method of claim 6 , wherein each input vector v i and v j comprises one of raw node features and computed node embeddings.

8 . The method of claim 1 , wherein the updated adjacency matrix is learned by minimizing a joint loss function based on = pred + , where pred is a prediction loss based on a downstream task and is a graph regularization loss.

9 . The method of claim 8 , wherein the graph regularization loss is defined by:
=αΩ( A,X )+ f ( A )

where α is a non-negative hyperparameter, A is an adjacency matrix, X is a feature matrix, and Ω(A,X) is a smoothness loss.

10 . The method of claim 8 , wherein is based on a Frobenius norm of the updated adjacency matrix.

11 . The method of claim 1 , wherein the initial noisy graph topology is obtained from one of an original data graph and a graph constructed using a k-nearest neighbors (kNN) strategy, the graph is based on sequential data or a feature matrix of a corresponding application.

12 . The method of claim 11 , further comprising selecting an iteration of the updated adjacency matrix as a replacement for the initial noisy graph topology.

13 . The method of claim 12 , wherein the natural language processing is performed for a call center task.

14 . The method of claim 1 , wherein the repeating the generating, producing, and feeding back operations continues until a corresponding updated adjacency matrix designated as A (t) is sufficiently similar to a corresponding optimized graph for use in prediction based on a threshold ε to derive a final graph à (t) .

15 . The method of claim 14 , wherein the final graph à (t) is derived based on:
à (t) =λL (0) +(1−λ){η f ( A (t) )+(1−η) f ( A (1) )}

where η is a hyperparameter, λ is a hyperparameter used to balance a trade-off between the updated adjacency matrix A (t) and the initial noisy graph topology designated as A (0) , L (0) is a normali

Assignees

• IBM
• Rensselaer Polytech Inst

Inventors

• Wu Lingfei
• Chen Yu
• Zaki Mohammed J

Classifications

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06F18/23213

  with fixed number of clusters, e.g. K-means clustering · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title