Automatic labeling of telecommunication network data to train supervised machine learning

Patent metadata

Field	Value
Publication number	US-2019303726-A1
Application number	US-201916443948-A
Country	US
Kind code	A1
Filing date	Jun 18, 2019
Priority date	Mar 9, 2018
Publication date	Oct 3, 2019
Grant date	—

Abstract

Systems and methods include obtaining network data including first data of devices and services in the network, Performance Monitoring (PM) data associated with the devices and services and with associated timestamps, and second data including any of tickets, alarms, and events affecting some of the devices and services and with associated timestamps; obtaining one or more target events from the second data based on associated operational impact in the network; determining the PM data that is statistically correlated with the one or more target events; determining the statistically correlated PM data over a corresponding time based on the associated timestamps of the PM data and the one or more target events; and providing labels for the determined statistically correlated PM data with an associated label based on the associated target event of the one or more target events.

First claim

What is claimed is:

1 . A system comprising:
a processor; and memory storing instructions that, when executed, cause the processor to
obtain network data including first data of devices and services in the network, Performance Monitoring (PM) data associated with the devices and services and with associated timestamps, and second data including any of tickets, alarms, and events affecting some of the devices and services and with associated timestamps,
obtain one or more target events from the second data based on associated operational impact in the network,
determine the PM data that is statistically correlated with the one or more target events,
determine the statistically correlated PM data over a corresponding time based on the associated timestamps of the PM data and the one or more target events, and
provide labels for the determined statistically correlated PM data with an associated label based on the associated target event of the one or more target events.

2 . The system of claim 1 , wherein the memory storing instructions that, when executed, cause the processor to
utilize a set of labeled data based on the provided labels to train a machine learning process.

3 . The system of claim 1 , wherein the memory storing instructions that, when executed, cause the processor to
subsequent to training a machine learning process with a set of labeled data based on the provided labels, obtain second PM data based on current operation of the network, process the second PM data via the machine learning process, and obtain predictions from the machine learning process based on labels associated with the set of labeled data.

4 . The system of claim 1 , wherein the statistical correlation includes measuring correlation of the PM data at a same time as each of the one or more target events and measuring the correlation of the PM data for prior time bins as each of the one or more target events.

5 . The system of claim 1 , wherein the network includes any of optical network elements, Time Division Multiplexing (TDM) network elements, Wavelength Division Multiplexing (WDM) network elements, and packet network elements.

6 . The system of claim 1 , wherein the devices in the network include a plurality of disparate types of devices from a plurality of equipment vendors.

7 . The system of claim 1 , wherein the associated label is based on one or more of a risk assessment of network equipment, service assurance, and application Quality of Experience (QoE).

8 . A method comprising:
obtaining network data including first data of devices and services in the network, Performance Monitoring (PM) data associated with the devices and services and with associated timestamps, and second data including any of tickets, alarms, and events affecting some of the devices and services and with associated timestamps; obtaining one or more target events from the second data based on associated operational impact in the network; determining the PM data that is statistically correlated with the one or more target events; determining the statistically correlated PM data over a corresponding time based on the associated timestamps of the PM data and the one or more target events; and providing labels for the determined statistically correlated PM data with an associated label based on the associated target event of the one or more target events.

9 . The method of claim 8 , further comprising
utilizing a set of labeled data based on the provided labels to train a machine learning process.

10 . The method of claim 8 , further comprising
subsequent to training a machine learning process with a set of labeled data based on the provided labels, obtaining second PM data based on current operation of the network; processing the second PM data via the machine learning process; and obtaining predictions from the machine learning process based on labels associated with the set of labeled data.

11 . The method of claim 8 , wherein the determining statistical correlation includes measuring correlation of the PM data at a same time as each of the one or more target events and measuring the correlation of the PM data for prior time bins as each of the one or more target events.

12 . The method of claim 8 , wherein the network includes any of optical network elements, Time Division Multiplexing (TDM) network elements, Wavelength Division Multiplexing (WDM) network elements, and packet network elements.

13 . The method of claim 8 , wherein the devices in the network include a plurality of disparate types of devices from a plurality of equipment vendors.

14 . The method of claim 8 , wherein the associated label is based on one or more of a risk assessment of network equipment, service assurance, and application Quality of Experience (QoE).

15 . A non-transitory computer-readable medium comprising instructions for automatically labeling data from a telecommunications network, wherein the instructions, when executed, cause a processor to perform the steps of:
obtaining network data including first data of devices and services in the network, Performance Monitoring (PM) data associated with the devices and services and with associated timestamps, and second data including any of tickets, alarms, and events affecting some of the devices and services and with associated timestamps; obtaining one or more target events from the second data based on associated operational impact in the network; determining the PM data that is statistically correlated with the one or more target events; determining the statistically correlated PM data over a corresponding time based on the associated timestamps of the PM data and the one or more target events; and providing labels for the determined statistically correlated PM data with an associated label based on the associated target event of the one or more target events.

16 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed, cause a processor to perform the steps of
utilizing a set of labeled data based on the provided labels to train a machine learning process.

17 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed, cause a processor to perform the steps of
subsequent to training a machine learning process with a set of labeled data based on the provided labels, obtaining second PM data based on current operation of the network; processing the second PM data via the machine learning process; and obtaining predictions from the machine learning process based on labels associated with the set of labeled data.

18 . The non-transitory computer-readable medium of claim 15 , wherein the determining statistical correlation includes measuring correlation of the PM data at a same time as each of the one or more target events and measuring the correlation of the PM data for prior time bins as each of the one or more target events.

19 . The non-transitory computer-readable medium of claim 15 , wherein the network includes any of optical network elements, Time Division Multiplexing (TDM) network elements, Wavelength Division Multiplexing (WDM) network elements, and packet network elements.

20 . The non-transitory computer-readable medium of claim 15 , wherein the associated label is based on one or more of a risk assessment of network equipment, service assurance, and application Quality of Experience (QoE).

Assignees

• Ciena Corp

Inventors

• Côté David
• Triplet Thomas

Classifications

• G06F18/2178 Primary

  based on feedback of a supervisor · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L41/5009

  Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF] · CPC title

• G06K9/6263 Primary

  Physics · mapped topic