Action recommendation engine (ARE) for network operations center (NOC) solely from raw un-labeled data

Patent metadata

Field	Value
Publication number	US-11637742-B2
Application number	US-202117166383-A
Country	US
Kind code	B2
Filing date	Feb 3, 2021
Priority date	Feb 3, 2021
Publication date	Apr 25, 2023
Grant date	Apr 25, 2023

Abstract

Systems, methods, and computer-readable media are provided for recommending actions to be taken in a network for optimizing or improving the operability of the network. A method, according to one implementation, includes a first step of receiving raw, unprocessed data that is obtained directly from one or more network elements of a network. The method includes second step of determining one or more remedial actions using a direct association between the raw, unprocessed data and the one or more remedial actions.

First claim

What is claimed is:

1. A non-transitory computer-readable medium configured to store computer logic having instructions that, when executed, enable a processing device to:
receive raw, unprocessed data obtained directly from one or more network elements of a network, and
determine one or more remedial actions using a direct association between the raw, unprocessed data and the one or more remedial actions, wherein the direct association is based on offline Reinforcement Learning (RL) that analyzed one or more of historical Network Operations Center (NOC) actions and operations on a network simulator.

2. The non-transitory computer-readable medium of claim 1 , wherein determining the one or more remedial actions is performed without determining a state of the one or more network elements.

3. The non-transitory computer-readable medium of claim 1 , wherein determining the one or more remedial actions includes utilizing an Action Recommendation Engine (ARE).

4. The non-transitory computer-readable medium of claim 3 , wherein the instructions further enable the processing device to:
receive a recommendation from the ARE regarding how, when, and where the one or more remedial actions are to be conducted on the network, and
leverage the recommendation to enable manual execution of the one or more remedial actions in the network.

5. The non-transitory computer-readable medium of claim 3 , wherein the instructions further enable the processing device to utilize the ARE to predict actions executed by a Network Operations Center (NOC) based on the raw, unprocessed data.

6. The non-transitory computer-readable medium of claim 1 , wherein the offline RL that analyzed the historical NOC actions includes evaluations of historical NOC actions not taken via exploration.

7. The non-transitory computer-readable medium of claim 6 , wherein the exploration includes
traversing historical time-series where context and action data is already pre-collected, computing the reward after each historical action, and updating the offline RL accordingly.

8. The non-transitory computer-readable medium of claim 1 , wherein the offline RL that analyzed operations on the network simulator includes validation of the learned direct association on a real network prior to use in production.

9. The non-transitory computer-readable medium of claim 1 , wherein the raw, unprocessed data is un-labeled and a state of the network is unknown.

10. The non-transitory computer-readable medium of claim 1 , wherein the raw, unprocessed data includes one or more of Performance Monitoring (PM) data, margin information, alarms, Quality of Service (QoS) information, Quality of Experience (QoE) information, configuration information, fiber cut information, and fault information.

11. The non-transitory computer-readable medium of claim 1 , wherein the one or more remedial actions include one or more of adjusting launch power at an amplifier, adjusting channel power at a Wavelength Selective Switch (WSS), adjusting a modulation scheme at an optical receiver, rebooting a card, cleaning or repairing a fiber, utilizing a protection path, adding bandwidth, defragmenting wavelengths across the network, running an Optical Time Domain Reflectometry (OTDR) trace, re-provisioning unprotected services after a loss of signal, adjusting Open Shortest Path First (OSPF) costs, re-routing Internet Protocol (IP) and Multi-Protocol Label Switching (MPLS) tunnels, modifying Border Gateway Protocol (BGP) routes, re-routing services based on utilization, auto-scaling Virtual Network Functions (VNFs), adjusting alarm thresholds, adjusting timer thresholds, clearing upstream alarms, fixing inventory, and upgrading software.

12. The non-transitory computer-readable medium of claim 1 , wherein the instructions further enable the processing device to collect data related to remedial actions conducted on the network, the data related to remedial actions being collected from one or more of shelf processor logs, command logs, a Network Management System (NMS) database, and Network Operations Center (NOC) tickets.

13. The non-transitory computer-readable medium of claim 1 , wherein the instructions further enable the processing device to learn a representation of a network state by observing hidden layers.

14. The non-transitory computer-readable medium of claim 1 , wherein the network is modeled in a simulated network environment.

15. The non-transitory computer-readable medium of claim 14 , wherein the instructions further enable the processing device to utilize a Reinforcement Learning (RL) technique to determine the one or more remedial actions of the simulated network environment and transfer the one or more remedial actions to an actual network.

16. The non-transitory computer-readable medium of claim 15 , wherein the instructions further enable the processing device to:
train RL agents with initial non-zero exploration in the simulated network environment, and
transfer pre-trained RL results from the simulated network environment to the actual network.

17. A system comprising:
one or more processing devices; and
a memory device configured to store computer logic having instructions that, when executed, enable the one or more processing devices to
receive raw, unprocessed data obtained directly from one or more network elements of a network, and
determine one or more remedial actions using a direct association between the raw, unprocessed data and the one or more remedial actions, wherein the direct association is based on offline Reinforcement Learning (RL) that analyzed one or more of historical Network Operations Center (NOC) actions and operations on a network simulator.

18. The system of claim 17 , wherein the instructions further enable the one or more processing devices to utilize Machine Learning (ML) to reproduce actions of a Network Operations Center (NOC) in communication with the network.

19. A method comprising the steps of:
receiving raw, unprocessed data obtained directly from one or more network elements of a network; and
determining one or more remedial actions using a direct association between the raw, unprocessed data and the one or more remedial actions, wherein the direct association is based on offline Reinforcement Learning (RL) that analyzed one or more of historical Network Operations Center (NOC) actions and operations on a network simulator.

20. The method of claim 19 , wherein the step of determining the one or more remedial actions includes utilizing an Action Recommendation Engine (ARE).

Assignees

• Ciena Corp

Inventors

• Côté David
• Shirmohammadi Shervin
• Mohammed Shady A
• Altamimi Sa'Di
• Altamimi Basel

Classifications

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N20/00

  Machine learning · CPC title