Graph neural network and reinforcement learning techniques for connection management

Patent metadata

Field	Value
Publication number	US-12375968-B2
Application number	US-202117561563-A
Country	US
Kind code	B2
Filing date	Dec 23, 2021
Priority date	Jun 30, 2021
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

The present disclosure provides connection management techniques based on graph neural networks (GNN) and deep reinforcement learning (DRL) to optimize user association and load balancing. A graph structure of a communication network is considered for the GNN architecture and DRL is used to learn parameters of the GNN algorithm/model. Connection management is defined as a combinatorial graph optimization problem, and the DRL mechanism uses the underlying graph to learn weights of the GNN for an optimal user connections or associations. The connection management techniques can consider local network features to make better decisions to balance network traffic load while network throughput is also maximized. Implementations are provided based on edge computing frameworks include the Open RAN (O-RAN) architecture. Other embodiments may be described and/or claimed.

First claim

The invention claimed is:

1. An apparatus comprising:
memory circuitry;
computer-readable instructions;
at least one programmable circuit to be programmed based on the computer-readable instructions to:
identify, after detection of a connection event, communication equipment instances (CEs) of a set of CEs and communication links of a set of communication links, the communication links between at least pairs of CEs in the set of CEs;
generate an initial graph including an initial arrangement of edges among a set of nodes, the set of nodes representative of respective CEs of the set of CEs, the edges in the initial arrangement of edges representative of respective communication links of the set of communication links;
cause a graph neural network (GNN) to determine a set of candidate graphs based on a set of node features, respective ones of the candidate graphs including corresponding candidate arrangements of edges between corresponding ones of the set of nodes, the candidate arrangements different than the initial arrangement;
determine respective quality (Q) values of the candidate graphs based on a Q function, the Q function based on a reinforcement learning model;
select at least one of the candidate graphs from among the set of candidate graphs based on the Q values, the selected at least one of the candidate graphs including an output arrangement of edges between corresponding ones of the nodes; and
generate connection management (CM) instructions to reconfigure the set of communication links between one or more CEs of the set of CEs based on the output arrangement of edges; and

interface circuitry to send the CM instructions to the one or more CEs in the set of CEs, the CM instructions to cause the one or more CEs to establish one or more communication links with other CEs in the set of CEs.

2. The apparatus of claim 1 , wherein a first Q value for a first candidate graph in the set of candidate graphs is an expected reward value associated with rearrangement of the set of communication links according to a first one of the candidate arrangements corresponding to the first candidate graph.

3. The apparatus of claim 2 , wherein the GNN includes a plurality of GNN layers, the plurality of GNN layers including at least an input layer, an output layer, and at least one hidden layer disposed between the input layer and the output layer, and the output layer of the GNN is an input to the Q function.

4. The apparatus of claim 1 , wherein, to determine the set of candidate graphs, one or more of the at least one programmable circuit is to:
identify, based on the initial graph, a current state of a communication network including the set of CEs; and
for a first candidate graph in the set of candidate graphs:
determine respective Q values for the edges in a first one of the candidate arrangements, the first one of the candidate arrangements corresponding to the first candidate graph; and
combine the respective Q values to obtain the Q value of the first candidate graph.

5. The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to:
operate the GNN based on at least one of throughput of a communication network including the set of CEs, coverage of the communication network, or load balance among the set of CEs.

6. The apparatus of claim 1 , wherein the set of node features includes data rates of the set of CEs and channel capacities of the set of CEs, and wherein:
the interface circuitry is to obtain network metrics from individual CEs of the set of CEs; and
one or more of the at least one programmable circuit is to determine the channel capacities and the data rates based on the obtained network metrics.

7. The apparatus of claim 6 , wherein the network metrics include one or more measurement reports, wherein a measurement report of the one or more measurement reports includes at least one signal or channel measurement, a CE identifier of a CE that performed the at least one signal or channel measurement, and an identifier of a coverage area in which the CE performed the at least one signal or channel measurement.

8. The apparatus of claim 7 , wherein the network metrics include at least one of respective CE rates of the individual CEs, respective CE spectral efficiency metrics of the individual CEs, respective CE resource utilization of the individual CEs, a total resource utilization associated with the set of CEs, respective bandwidth utilization at the individual CEs, a total bandwidth utilization associated with the set of CEs, or CE status information that indicated whether ones of the individual CEs are in an active mode, an inactive mode, or an idle mode.

9. The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to:
cause storage of connection events in a connection event queue; and
process the queued connection events in a first in first out manner.

10. The apparatus of claim 9 , wherein one or more of the at least one programmable circuit is to process a number of the queued connection events in parallel.

11. The apparatus of claim 1 , wherein the set of CEs includes a set of network access nodes (NANs) and a set of user equipment (UEs), individual NANs of the set of NANs provide network connectivity to one or more UEs of the set of UEs, and the communication links in the set of communication links are between ones of the UEs and ones of the NANs.

12. The apparatus of claim 11 , wherein one or more of the at least one programmable circuit is to generate, as the initial graph, a NAN-NAN adjacency graph based on locations of the individual NANs in the set of NANs.

13. The apparatus of claim 12 , wherein to one or more of the at least one programmable circuit is to generate the NAN-NAN adjacency graph to include a first NAN in the set of NANs serving a UE that is a subject of the detected connection event and a subset of second NANs in the set of NANs within a predetermined distance from the first NAN.

14. The apparatus of claim 13 , wherein one or more of the at least one programmable circuit is to generate the NAN-NAN adjacency graph based on a score calculated based on network metrics collected from individual UEs of the set of UEs.

15. The apparatus of claim 11 , wherein the apparatus corresponds to a Radio Access Network (RAN) Intelligent Controller (RIC) in an Open RAN (O-RAN) framework, the computer-readable instructions correspond to an xApp operated by the RIC, and the individual NANs are at least one of O-RAN distributed units (DUs) or O-RAN remote units (RUs).

16. At least one computer-readable medium comprising instructions to cause at least one programmable circuit to at least:
identify, after detection of a connection event, communication equipment instances (CEs) of a set of CEs and communication links of a set of communication links, the communication links between at least pairs of CEs in the set of CEs;
generate an initial graph including an initial arrangement of edges among a set of nodes, the set of nodes representative of respective CEs of the set of CEs, the edges in the initial arrangement of edges representative of respective communication links of the set of communication links;
cause a graph neural network (GNN) to determine a set of candidate graphs based on a set of node features, respective ones of the candidate graphs including corresponding candidate arrangements of edges between corresponding ones of the set of nodes, the candidate arrangements different than the initial arrangement;
determine respective quality (Q) values of the candidate graphs based on a reinforceme

Assignees

• Intel Corp

Inventors

• Orhan Oner
• Swamy Vasuki Narasimha
• Nassar Marcel
• Nikopour Hosein
• Talwar Shilpa

Classifications

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H04W28/086 Primary

  among access entities · CPC title

• H04W28/0925

  using policies · CPC title

• G06N3/08

  Learning methods · CPC title