Application quality of experience evaluator for enhancing subjective quality of experience

Patent metadata

Field	Value
Publication number	US-10454989-B2
Application number	US-201615047728-A
Country	US
Kind code	B2
Filing date	Feb 19, 2016
Priority date	Feb 19, 2016
Publication date	Oct 22, 2019
Grant date	Oct 22, 2019

Abstract

A method to enhance a subjective quality of experience for an application may include receiving network performance data, the data representing at least one observable application characteristic, and the subjective quality of experience (QoE) survey data. The method may further include generating at least one perception model which relates the data representing at least one observable application characteristic and the network performance data, and determining a QoE model which relates the subjective QoE survey data and the data representing at least one observable application characteristic. The method may further include inverting the at least one perception model and the QoE model to obtain a relationship between network performance parameters and the at least one observable application characteristic, and adjusting network parameters based on the at least one inverted perception model and inverted QoE model.

First claim

What is claimed is:

1. A method for enhancing a subjective quality of experience (QoE) for a user application, comprising:
receiving network performance data, wherein the network performance data includes measurable network key performance indicators;
receiving data representing at least one observable user application characteristic;
receiving subjective QoE survey data;
generating at least one perception model which relates the data representing at least one observable user application characteristic and the network performance data;
determining a QoE model which relates the subjective QoE survey data and the data representing at least one observable user application characteristic;
determining at least one user application characteristic value that improves the QoE for the user application based on an inverse function of the QoE model;
determining network performance parameters that map to the at least one user application characteristic value based on an inverse function for the at least one perception model; and
adjusting a region of a network based on the determined network performance parameters.

2. The method of claim 1 , wherein generating at least one perception model includes ensemble learning, wherein ensemble learning further comprises:
generating ensemble models relating each observable user application characteristic to network performance parameters using a training set of network performance data, wherein each of the ensemble models is based on the execution of a different regression algorithm;
determining errors for each of the generated ensemble models based on a cross validation set of network performance data;
generating a final ensemble model based on a weighted combination of the ensemble models; and
validating the final ensemble model based on a test set of network performance data.

3. The method of claim 2 , wherein generating ensemble models further comprises:
applying at least one of linear regression, polynomial regression of at least one of order 2 or order 3 , lasso regression, ridge regression, elastic regression, or a generalized additive model.

4. The method of claim 2 , wherein generating the final ensemble model further comprises:
assigning weights associated with a weighted combination based on determined errors for each of the generated ensemble models.

5. The method of claim 4 , wherein assigning the weights further comprises:
iteratively calculating the weights using a least squares minimization until the weights converge.

6. The method of claim 1 , wherein determining the QoE model further comprises:
mapping dichotomous comparisons from the subjective QoE survey data to a continuous QoE score as a function of each user application characteristic.

7. The method of claim 6 , further comprising:
determining the continuous QoE score based on a Bradley-Terry-Luce model.

8. The method of claim 6 , further comprising
determining a mean opinion score (MOS) by normalizing the continuous QoE score.

9. A network device, comprising:
an interface that communicates with a network;
a memory configured to store instructions; and
a processor, coupled to the interface and the memory, wherein the processor is configured to execute the instructions stored in the memory to:
receive network performance data, wherein the network performance data includes measurable network key performance indicators,
receive data representing at least one observable user application characteristic,
receive subjective quality of experience (QoE) survey data,
generate at least one perception model which relates the data representing at least one observable user application characteristic and the network performance data,
determine a QoE model which relates the subjective QoE survey data and the data representing at least one observable user application characteristic,
determine at least one user application characteristic value that improves the QoE for a user application based on an inverse function of the QoE model;
determine network performance parameters that map to the at least one user application characteristic value based on an inverse function for the at least one perception model, and
adjust a region of a network based on the determined network parameters.

10. The network device of claim 9 , wherein the instructions to generate at least one perception model includes instructions for ensemble learning, further causing the processor to:
generate ensemble models relating each observable user application characteristic to network performance parameters using a training set of network performance data, wherein each of the ensemble models is based on the execution of a different regression algorithm,
determine errors for each of the generated ensemble models based on a cross validation set of network performance data,
generate a final ensemble model based on a weighted combination of the ensemble models, and
validate the final ensemble model based on a test set of network performance data.

11. The network device of claim 10 , wherein the instructions to generate ensemble models further causes the processor to:
apply at least one of linear regression, polynomial regression of at least one of order 2 or order 3 , lasso regression, ridge regression, elastic regression, or a generalized additive model.

12. The network device of claim 10 , wherein the instructions to generate a final ensemble model further causes the processor to:
assign weights associated with a weighted combination based on determined errors for each of the generated ensemble models.

13. The network device of claim 12 , wherein the instructions to assign weights further cause the processor to:
iteratively calculate the weights using a least squares minimization until the weights converge.

14. The network device of claim 9 , wherein the instructions to determine the QoE model further cause the processor to:
map dichotomous comparisons from the subjective QoE survey data to a continuous QoE score as a function of each user application characteristic.

15. The network device of claim 14 , wherein the memory comprises instructions causing the processor to:
determine the continuous QoE score based on a Bradley-Terry-Luce model.

16. The network device of claim 14 , wherein the memory comprises instructions causing the processor to:
determine a mean opinion score (MOS) by normalizing the continuous QoE score.

17. A non-transitory computer-readable medium comprising instructions, which, when executed by a processor, causes the processor to:
receive network performance data, wherein the network performance data includes measurable network key performance indicators;
receive data representing at least one observable user application characteristic;
receive subjective quality of experience (QoE) survey data;
generate at least one perception model which relates the data representing at least one observable user application characteristic and the network performance data;
determine a QoE model which relates the subjective QoE survey data and the data representing at least one observable user application characteristic;
determine at least one user application characteristic value that improves the QoE for a user application based on an inverse function of the QoE model;
determine performance parameters that map to the at least one user application characteristic value based on an inverse function for the at least one perception model; and
adjust a region of a network based on the determined network performance parameters.

Assignees

• Verizon Patent & Licensing Inc

Inventors

• Ouyang Ye
• Becht Carol
• Iyer Krishna Pichumani

Classifications

• H04L65/80 Primary

  Responding to QoS · CPC title

• H04L41/5067

  Customer-centric QoS measurements · CPC title

• H04L43/08

  Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters · CPC title

• H04L41/142 Primary

  using statistical or mathematical methods · CPC title