Systems and methods for evaluating a user experience in a network based on performance indicators

Patent metadata

Field	Value
Publication number	US-11240131-B2
Application number	US-201916387012-A
Country	US
Kind code	B2
Filing date	Apr 17, 2019
Priority date	Apr 17, 2019
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

A device may collect network performance data associated with a user equipment of a network. The network performance data may include information associated with a plurality of performance indicators of the network. The device may process information associated with a first portion of the plurality of performance indicators to determine a first performance category experience score, and information associated with a second portion of the plurality of performance indicators to determine a second performance category experience score. The device may process the first performance category experience score and the second performance category experience score to determine a network experience score. The device may determine whether the network experience score satisfies a threshold value. The device may perform one or more actions based on determining that the network experience score satisfies the threshold value.

First claim

What is claimed is:

1. A method, comprising:
collecting, by a device, network performance data associated with a user equipment of a network,
wherein the network performance data associated with the user equipment of the network includes information associated with a plurality of performance indicators of the network,
wherein the plurality of performance indicators are correlated to a user experience of a user of the user equipment, and
wherein a first portion of the plurality of performance indicators are associated with a first performance category and a second portion of the plurality of performance indicators are associated with a second performance category;

processing, by the device and using a machine learning model:
information associated with the first portion of the plurality of performance indicators to determine a first performance category experience score for the user, and
information associated with the second portion of the plurality of performance indicators to determine a second performance category experience score for the user,
wherein the information associated with the first portion of the plurality of performance indicators and the information associated with the second portion of the plurality of performance indicators are normalized using an area under a curve algorithm prior to being processed by the machine learning model;

processing, by the device, the first performance category experience score and the second performance category experience score to determine a network experience score for the user,
wherein the network experience score includes a first network experience score for voice communication for the user and a second network experience score for data communication for the user;

determining, based on weightings of the first network experience score for voice communication and the second network experience score for data communication, an overall experience score;
determining, by the device, whether the network experience score for the user satisfies a threshold value,
wherein the network experience score satisfying the threshold value indicates an issue with the user experience of the user; and

performing, by the device, one or more actions based on determining to address the issue with the user experience of the user.

2. The method of claim 1 , wherein the one or more actions include one or more of:
transmitting a notification, to the user equipment, providing information relating to the issue, or
transmitting a notification, to a customer support terminal associated with the network, providing information relating to the issue.

3. The method of claim 1 , wherein a degree of correlation between a performance indicator of the plurality of performance indicators and the user experience is determined using a model.

4. The method of claim 1 , where the threshold value is a first threshold value,
wherein a performance indicator of the plurality of performance indicators is to be included in the plurality of performance indicators if a degree of correlation between the performance indicator and the user experience satisfies a second threshold value.

5. The method of claim 1 , wherein a first performance indicator of the plurality of performance indicators is correlated to the user experience when the first performance indicator is correlated to a second performance indicator of an application of the user equipment.

6. The method of claim 1 , wherein processing the first performance category experience score and the second performance category experience score is performed with a second machine learning model.

7. The method of claim 1 , wherein the first portion of the performance indicators or the second portion of the performance indicators relates to voice over Long-Term Evolution (VoLTE) communications or voice over 5G (Vo5G) communications, and
wherein the first portion of the performance indicators or the second portion of the performance indicators includes one or more of:
a session establishment effectiveness ratio (SEER),
a call drop rate,
a call connection rate, or
a call setup failure rate.

8. A device, comprising:
one or more memories; and
one or more processors, coupled to the one or more memories, to:
collect network performance data associated with a user equipment of a network,
wherein the network performance data includes information associated with a plurality of performance indicators of the network,
wherein the plurality of performance indicators are correlated to a user experience of a user of the user equipment, and
wherein a first portion of the plurality of performance indicators are associated with a first performance category and a second portion of the plurality of performance indicators are associated with a second performance category;

process, with a first machine learning model:
information associated with the first portion of the plurality of performance indicators to determine a first performance category experience score for the user, and
information associated with the second portion of the plurality of performance indicators to determine a second performance category experience score for the user,
wherein the information associated with the first portion of the plurality of performance indicators and the information associated with the second portion of the plurality of performance indicators are normalized using an area under a curve algorithm prior to being processed with the first machine learning model;

process, with a second machine learning model, the first performance category experience score and the second performance category experience score to determine a network experience score for the user,
wherein the network experience score includes a first network experience score for voice communication for the user and a second network experience score for data communication for the user;

determine, based on weightings of the first network experience score for voice communication and the second network experience score for data communication, an overall experience score;
determine whether the network experience score for the user satisfies a threshold value,
wherein the network experience score satisfying the threshold value indicates a poor quality associated with the user experience of the user; and

perform one or more actions based on determining that the network experience score for the user satisfies the threshold value to address the poor quality associated with the user experience.

9. The device of claim 8 , wherein the one or more processors, when performing the one or more actions, are to perform one or more of:
transmit a notification, to the user equipment, providing information relating to the poor quality associated with the user experience,
transmit a notification, to a customer support terminal associated with the network, providing information relating to the poor quality associated with the user experience, or
update at least one of the first machine learning model or the second machine learning model.

10. The device of claim 8 , wherein the one or more processors, when performing the one or more actions, are to:
receive an update to the network experience score from the user to obtain an updated network experience score; and
update at least one of the first machine learning model or the second machine learning model based on the updated network experience score.

11. The device of claim 8 , wherein the first machine learning model is a convolutional neural network model and the second machine learning model is a feedforward neural network model.

12. The device of claim 8 , wherein the user experience of the user relat

Assignees

• Verizon Patent & Licensing Inc

Inventors

• Su Le
• Ouyang Ye
• Lu Wenyuan
• Iyer Krishna Pichumani
• Abbas Khurram
• Rice Steven F
• Brainard Garrison William
• Woods Dwight Ellis

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• H04L43/065 Primary

  related to network devices · CPC title

• H04L41/5067 Primary

  Customer-centric QoS measurements · CPC title