Methods, systems and computer readable media for predicting risk in network elements using machine learning

Patent metadata

Field	Value
Publication number	US-11337087-B2
Application number	US-202016803688-A
Country	US
Kind code	B2
Filing date	Feb 27, 2020
Priority date	May 16, 2019
Publication date	May 17, 2022
Grant date	May 17, 2022

Abstract

A method, system and computer-readable medium where a weighted composite quality index having a plurality of components for a network element is identified. A historical baseline value from historical data for each component is determined, and a deviation from the historical baseline values is measured. A risk level for the deviation is assigned. A loss score for the measured components is computed by mapping the risk level to a numerical score. An aggregated risk score based on a sum of weighted risk scores for each of the components is computed. An expected risk score based on probabilities associated with the aggregated risk score is determined by computing future probabilities of each risk level at the network element based on a trained machine learning model.

First claim

What is claimed:

1. A method comprising:
identifying, for a plurality of network elements, a respective composite quality index (CQI) for each of data quality and voice quality, wherein each respective CQI has a respective plurality of components and a weight associated with each component in the respective plurality of components, wherein the plurality of components of the CQI for voice quality comprises a number of dropped calls, blocked calls, network driven churn, call establishment failure, echo, distortion, and background noise, and wherein the plurality of components of the CQI for data quality comprises packet loss, latency, and jig;
determining a historical baseline value from historical data for each component in the respective plurality of components for each CQI, resulting in historical baseline values;
determining an abnormality from the historical baseline values in at least one of the respective plurality of components for at least one of the CQIs, resulting in an at least one abnormality;
assigning a respective risk level for each abnormality included in the at least one abnormality;
mapping the assigned respective risk level for each abnormality to a respective numerical score;
using a trained machine learning model, determining a respective probability of future risk levels for each component of the respective plurality of components for each CQI based on the mapping; and
based on the respective probability of future risk levels, determining which of a subset of the plurality of network elements is at risk of impacting customer quality of experience, wherein the determining of which of the subset of the plurality of network elements is at risk of impacting customer quality of experience is used to preempt poor customer experience by proactively identifying network elements that are more at risk of impacting the customer quality of experience.

2. The method of claim 1 , wherein the subset includes an eNodeB.

3. The method of claim 1 , wherein the respective risk level for each abnormality comprises a risk level selected from a group comprising no risk, low risk, medium risk, high risk, and critical risk.

4. The method of claim 1 , wherein the determining of the respective probability of future risk levels comprises:
inputting data from one or more data sources into the trained machine learning model, wherein the data comprises:
past values of an abnormality of each member of the respective plurality of components of each of the CQIs;
a historical baseline value of each member of the respective plurality of components of each of the CQIs; and
a future baseline value for each member of the respective plurality of components of each of the CQIs for a period of time.

5. The method of claim 4 , further comprising creating the trained machine learning model by:
computing the historical baseline value of each member of the respective plurality of components of each of the CQIs;
extracting raw and derived time series for risk related performance metrics and load and traffic metrics for each network element of the plurality of network elements; and
inputting the historical baseline value of each member of the respective plurality of components of each of the CQIs into a risk labeler.

6. A system comprising:
a memory for storing computer instructions; and
a processor coupled with the memory, wherein the processor, responsive to executing the computer instructions, performs operations comprising:
identifying, for a plurality of network elements, a respective composite quality index (CQI) for each of data quality and voice quality, each of the CQIs having a respective plurality of components and a weight associated with each component in the respective plurality of components, wherein the plurality of components of the CQI for voice quality comprises a number of dropped calls, blocked calls, network driven churn, call establishment failure, echo, distortion, and background noise, and wherein the plurality of components of the CQI for data quality comprises packet loss, latency, and jig;
determining a historical baseline value from historical data for each component in the respective plurality of components for each CQI, resulting in historical baseline values;
determining an abnormality from the historical baseline values in at least one of the respective plurality of components for at least one of the CQIs, resulting in an at least one abnormality;
assigning a respective risk level for each abnormality included in the at least one abnormality;
mapping the assigned respective risk level for each abnormality to a respective numerical score;
using a trained machine learning model, calculating a respective probability of future risk levels for each component of the respective plurality of components for each CQI based on the mapping; and
based on the respective probability of future risk levels, determining which of a subset of the plurality of network elements is at risk of impacting customer quality of experience,
wherein the determining of which of the subset of the plurality of network elements is at risk of impacting customer quality of experience is used to preempt poor customer experience by proactively identifying network elements that are more at risk of impacting the customer quality of experience.

7. The system of claim 6 , wherein the subset includes an eNodeB.

8. The system of claim 6 , wherein the respective risk level for each abnormality comprises a risk level selected from a group comprising no risk, low risk, medium risk, high risk and critical risk.

9. The system of claim 6 , wherein the calculating of the respective probability of future risk levels comprises:
inputting data from one or more data sources into the trained machine learning model, wherein the data comprises:
past values of an abnormality of each member of the respective plurality of components of each of the CQIs;
a historical baseline value of each member of the respective plurality of components of each of the CQIs; and
a future baseline value for each member of the respective plurality of components of each of the CQIs for a period of time.

10. The system of claim 9 , wherein the processor, responsive to executing the computer instructions, performs operations comprising creating the trained machine learning model by:
computing the historical baseline value of each member of the respective plurality of components of each of the CQIs;
extracting raw and derived time series for risk related performance metrics and load and traffic metrics for each network element of the plurality of network elements; and
inputting the historical baseline value of each member of the respective plurality of components of each of the CQIs into a risk labeler.

11. A tangible non-transitory computer-readable medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to perform operations comprising:
identifying, for a plurality of network elements, a respective composite quality index (CQI) for each of data quality and voice quality, wherein each respective CQI has a respective plurality of components and a weight associated with each component in the respective plurality of components, wherein the plurality of components of the CQI for voice quality comprises a number of dropped calls, blocked calls, network driven churn, call establishment failure, echo, distortion, and background noise, and wherein the plurality of components of the CQI for data quality comprises packet loss, latency, and jig;
determining a historical baseline value from historical data for each component in the respective plurality of components for each CQI, resulti

Assignees

• At & T Ip I Lp
• At & T Mobility Ii Llc

Inventors

• Deb Supratim
• Yan He
• Biswas Karunasish
• Notter Joseph Allen

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N20/00

  Machine learning · CPC title

• H04W24/04 Primary

  Arrangements for maintaining operational condition · CPC title

• H04W24/08

  Testing, {supervising or monitoring} using real traffic · CPC title

• H04W88/08

  Access point devices · CPC title