Multi-batch reinforcement learning via multi-imitation learning

Patent metadata

Field	Value
Publication number	US-12452733-B2
Application number	US-202217957960-A
Country	US
Kind code	B2
Filing date	Sep 30, 2022
Priority date	Oct 6, 2021
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

A server may receive a first traffic data and a second traffic data from a first base station and a second base station; obtain a first augmented traffic data for the first base station, based on the first traffic data and a subset data of the second traffic data; obtain a second augmented traffic data for the second base station, based on the second traffic data and a subset data of the first traffic data; obtain a first artificial intelligence (AI) model via imitation learning based on the first augmented traffic data; obtain a second AI model imitation learning based on the second augmented traffic data; obtain a generalized AI model via knowledge distillation from the first AI model and the second AI model; and predict a future traffic load of each of the first base station and the second base station based on the generalized AI model.

First claim

What is claimed is:

1. A method for performing traffic load balancing in a communication system, the method comprising:
receiving a first traffic data from a first base station;
receiving a second traffic data from a second base station;
obtaining a first augmented traffic data for the first base station, based on the first traffic data and a subset data of the second traffic data that is selected based on similarity between the first traffic data and the second traffic data;
obtaining a second augmented traffic data for the second base station, based on the second traffic data and a subset data of the first traffic data that is selected based on the similarity between the first traffic data and the second traffic data;
obtaining a first artificial intelligence (AI) model based on the first augmented traffic data;
obtaining a second AI model based on the second augmented traffic data;
obtaining a generalized AI model via knowledge distillation from the first AI model and the second AI model; and
predicting a traffic load of each of the first base station and the second base station based on the generalized AI model.

2. The method of claim 1 , wherein the obtaining the first augmented traffic data comprises:
obtaining an upper envelope function based on the first traffic data;
obtaining a sample selection ratio for a state-action pair of the second traffic data based on the upper envelope function; and
based on the sample selection ratio for the state-action pair of the second traffic data being greater than a predetermined sample selection ratio threshold, appending the state-action pair of the second traffic data to the first augmented traffic data.

3. The method of claim 2 , wherein the sample selection ratio represents a similarity between state-action pairs of the first traffic data and the state-action pair of the second traffic data.

4. The method of claim 2 , wherein the state-action pair comprises a state of the communication system, and a reward to be received as a result of taking an action at the state, and
wherein the state comprises at least one of an active user equipment (UE) number, an internet protocol (IP) throughput, a cell physical resource usage, and a speed of a download link, the action comprises a load balancing parameter that causes the state to be changed, and the reward comprises a minimum of IP throughput.

5. The method of claim 2 , wherein the obtaining the sample selection ratio comprises:
obtaining an estimated return of the state-action pair of the second traffic data; and
obtaining the sample selection ratio by dividing the estimated return of the state-action pair of the second traffic data by the upper envelope function.

6. The method of claim 1 , wherein the first AI model and the second AI model are obtained via imitation learning, and
wherein the obtaining the generalized AI model comprises:
computing a distillation loss between actions output from the first AI model and the second AI model and actions output from the generalized AI model, and
training the generalized AI model based on the distillation loss.

7. The method of claim 6 , wherein the obtaining the generalized AI model comprises:
computing a triplet loss based on a difference between identical tasks and a difference between different tasks, wherein the identical tasks are to predict the traffic load for a same base station, and the different tasks are to predict the traffic load for each of different base stations, and an identity of each of the identical tasks and the different tasks is provided by a task interference AI model;
combining the distillation loss and the triplet loss as an overall loss of the generalized AI model; and
training the generalized AI model until the overall loss converges or reaches a preset threshold value.

8. The method of claim 1 , further comprising:
transmitting the generalized AI model to the first base station to update the generalized AI model as a first updated AI model based on a system observation of the first base station; and
transmitting the generalized AI model to the second base station to update the generalized AI model as a second updated AI model based on a system observation of the second base station.

9. The method of claim 1 , wherein each of the first traffic data and the second traffic data comprises information of at least one of a number of active user equipment (UEs) per cell, a cell load ratio, and an internet protocol (IP) throughput per cell.

10. A server for performing traffic load balancing in a communication system, the server comprising:
a memory storing instructions, and
at least one processor configured to execute the instructions to:
receive a first traffic data from a first base station;

receive a second traffic data from a second base station;
obtain a first augmented traffic data for the first base station, based on the first traffic data and a subset data of the second traffic data that is selected based on similarity between the first traffic data and the second traffic data;
obtain a second augmented traffic data for the second base station, based on the second traffic data and a subset data of the first traffic data that is selected based on the similarity between the first traffic data and the second traffic data;
obtain a first artificial intelligence (AI) model based on the first augmented traffic data;
obtain a second AI model based on the second augmented traffic data;
obtain a generalized AI model via knowledge distillation from the first AI model and the second AI model; and
predict a traffic load of each of the first base station and the second base station based on the generalized AI model.

11. The server of claim 10 , wherein the at least one processor is further configured to execute the instructions to:
obtain an upper envelope function based on the first traffic data;
obtain a sample selection ratio for a state-action pair of the second traffic data based on the upper envelope function; and
based on the sample selection ratio being greater than a predetermined sample selection ratio threshold, append the state-action pair of the second traffic data to the first augmented traffic data.

12. The server of claim 11 , wherein the sample selection ratio represents a similarity between state-action pairs of the first traffic data and the state-action pair of the second traffic data.

13. The server of claim 11 , wherein the state-action pair comprises a state of the communication system, and a reward to be received as a result of taking an action at the state, and
wherein the state comprises at least one of an active user equipment (UE) number, an internet protocol (IP) throughput, and a cell physical resource usage, and a speed of a download link, the action comprises a load balancing parameter that causes the state to be changed, and the reward comprises a minimum of IP throughput.

14. The server of claim 11 , wherein the at least one processor is further configured to execute the instructions to:
obtain an estimated return of the state-action pair of the second traffic data; and
obtain the sample selection ratio by dividing the estimated return of the state-action pair of the second traffic data by the upper envelope function.

15. The server of claim 10 , wherein the first AI model and the second AI model are obtained via imitation learning, and
wherein the at least one processor is further configured to execute the instructions to:
compute a distillation loss between actions output from the first AI model and the second AI model and actions output from the generalized AI model, and

Assignees

• Samsung Electronics Co Ltd

Inventors

• Wu Di
• Li Tianyu
• Meger David
• Jenkin Michael
• Liu Xue
• Dudek Gregory Lewis

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N3/08

  Learning methods · CPC title

• H04W24/10

  Scheduling measurement reports {; Arrangements for measurement reports} · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title

• H04W24/02

  Arrangements for optimising operational condition · CPC title