Optimizing key allocation during roaming using machine learning

Patent metadata

Field	Value
Publication number	US-11910249-B2
Application number	US-202117391559-A
Country	US
Kind code	B2
Filing date	Aug 2, 2021
Priority date	Aug 2, 2021
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

Systems and methods are provided for optimizing resource consumption by bringing intelligence to the key allocation process for fast roaming. Specifically, embodiments of the disclosed technology use machine learning to predict which AP a wireless client device will migrate to next. In some embodiments, machine learning may also be used to select a subset of top neighbors from a neighborhood list. Thus, instead of allocating keys for each of the APs on the neighborhood list, key allocation may be limited to the predicted next AP, and the subset of top neighbors. In some embodiments, a reinforcement learning model may be used to dynamically adjust the size of the subset in order to optimize resources while satisfying variable client demand.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
learning migration behavior associated with a client device;
receiving a notification that comprises a current access point and a neighborhood list, wherein the current access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the current access point;
predicting an access point the client device will migrate to next based on the current access point and the learned client device migration behavior;
selecting a subset of neighbor access points from the neighborhood list based on the current access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning;
generating client keys for the predicted next access point and each of the subset of neighbor access points, wherein the client keys are cryptographic keys associated with the client device; and
propagating one of the client keys to each of the predicted next access point and the subset of neighbor access points.

2. The non-transitory computer-readable storage medium of claim 1 , wherein dynamically adjusting the size of the subset using machine learning comprises repeatedly performing the following:
selecting one or more integer values greater than or equal to zero, and less than or equal to the number of access points on the neighborhood list;
evaluating the reward for the selected values; and
choosing the evaluated value which gives the maximum reward.

3. The non-transitory computer-readable storage medium of claim 2 , wherein selecting the one or more integer values greater than or equal to zero, and less than or equal to the number of access points on the neighborhood list, comprises using an epsilon greedy policy.

4. The non-transitory computer-readable storage medium of claim 3 , wherein evaluating the reward for a selected value comprises:
rewarding successful fast roaming attempts; and
punishing failed fast roaming attempts.

5. The non-transitory computer-readable storage medium of claim 4 , wherein:
a successful fast roaming attempt comprises a client device connecting to an access point which has a client key cached; and
a failed fast roaming attempt comprises a client device connecting to an access point which:
the client device has connected to before; and
does not have a client key cached.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the computing system is a cloud-based key management service.

7. The non-transitory computer-readable storage medium of claim 1 , wherein an access point is in the neighborhood of the current access point when the access point's static path loss value relative to the current access point is below a threshold value.

8. The non-transitory computer-readable storage medium of claim 1 , wherein predicting the access point the client device will migrate to next comprises using a Markov Chain Model.

9. The non-transitory computer-readable storage medium of claim 8 , wherein selecting the subset of neighbor access points from the neighborhood list comprises using a Markov Chain Model.

10. The non-transitory computer-readable storage medium of claim 1 , wherein predicting the access point the client device will migrate to next comprises using a deep machine learning based Long Short-Term Memory (LSTM) model.

11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
learning migration behavior associated with a client device;
receiving a notification that comprises a current access point and a neighborhood list, wherein the current access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the current access point;
predicting a subset of neighbor access points from the neighborhood list based on the current access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning; and
generating client keys for the predicted next access point and each of the subset of neighbor access points.

12. The system of claim 11 , wherein dynamically adjusting the size of the subset using machine learning comprises:
selecting one or more integer values greater than zero and less than or equal to the number of access points on the neighborhood list;
evaluating the reward for the selected values;
choosing the evaluated value which gives the maximum reward; and
in response to deterioration of the reward for the chosen value, repeating the selecting, the evaluating, and the choosing, and so on.

13. The system of claim 12 , wherein evaluating the reward for a selected value comprises:
rewarding successful fast roaming iterations; and
punishing failed fast roaming iterations.

14. The system of claim 13 , wherein:
a successful fast roaming iteration comprises one or more successful fast roaming attempts, and zero failed fast roaming attempts;
a failed fast roaming iteration comprises one or more failed fast roaming attempts;
a successful fast roaming attempt comprises a client device connecting to an access point which has a client key cached; and
a failed fast roaming attempt comprises a client device connecting to an access point which:
the client device has connected to before; and
does not have a client key cached.

15. The system of claim 14 , wherein deterioration of the reward for a chosen value comprises at least one of the following:
the number of failed fast roaming iterations increasing relative to the number of successful fast roaming iterations; and
the number of access points at which client keys are cached, which are not connected to during a successful fast roaming iteration, increases.

16. A computer-implemented method comprising:
receiving, by a first access point, from a key management service, a cryptographic key associated with a client device, wherein the key management service operates to:
learn migration behavior associated with the client device;
receive a notification that comprises a second access point and a neighborhood list, wherein the second access point is the access point the client device is currently connected to and the neighborhood list is a list of access points in the neighborhood of the second access point;
predict a subset of neighbor access points from the neighborhood list based on the second access point and the learned client device migration behavior, wherein the size of the subset is dynamically adjusted using machine learning and the first access point is included in the subset;
generate cryptographic keys associated with the client device for each of the subset of neighbor access points; and
propagating one of the cryptographic keys associated with the client device to each of subset of neighbor access points; and

caching, by the first access point, the cryptographic key associated with the client device.

17. The computer-implemented method of claim 16 , wherein dynamically adjusting the size of the subset using machine learning comprises:
selecting one or more values between zero and the number of access points on the neighborhood list;
evaluating the reward for the selected values;
choosing

Assignees

• Hewlett Packard Entpr Dev Lp

Inventors

• Gupta Gopal
• Mishra Abhinesh
• Theogaraj Isaac
• Ganu Sachin
• Bandemer Bernd
• Tellado Jose

Classifications

• H04W36/00835

  Determination of neighbour cell lists · CPC title

• H04W36/0038 Primary

  of security context information · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title

• H04W12/041

  Key generation or derivation · CPC title