Method for computing environment specific baselines for metrics of user experience

Patent metadata

Field	Value
Publication number	US-11533243-B2
Application number	US-202017004945-A
Country	US
Kind code	B2
Filing date	Aug 27, 2020
Priority date	Jul 16, 2020
Publication date	Dec 20, 2022
Grant date	Dec 20, 2022

Abstract

Described embodiments provide systems, methods, and computer readable media for generating environment descriptors. A device having at least one process may identify a plurality of feature vectors. Each vector may describe a corresponding access to an application hosted on a server in one of a plurality of network environments and having a corresponding performance metric. The device may provide a performance model using the plurality of feature vectors and the corresponding performance metrics. The performance model may be used to determine expected performance metrics for at least a first network environment. A first environment descriptor of the expected performance metrics may be generated for at least the first network environment. The first environment descriptor of the expected performance metrics may be used to assess a measured performance metric or a second environment descriptor of a second network environment.

First claim

What is claimed is:

1. A method of generating environment descriptors, comprising:
identifying, by a device having at least one processor, a plurality of feature vectors each identifying a corresponding performance metric corresponding to access by one or more clients via at least one of a plurality of network environments to an application hosted on a server;
providing, by the device, a performance model using the plurality of feature vectors and the corresponding performance metrics;
determining, using the performance model, first expected performance metrics corresponding to access of the application by at least one first client via at least a first network environment;
generating, based at least on the first expected performance metrics, a first environment descriptor for at least the first network environment to be used by the at least one first client to access the application; and
determining a similarity metric between the first environment descriptor of the first expected performance metrics and a second environment descriptor of second expected metrics for a second network environment to be used by at least one second client.

2. The method of claim 1 , further comprising generating, using the first environment descriptor of the first expected performance metrics, a threshold metric for identifying an outlier for at least the first network environment.

3. The method of claim 1 , further comprising generating an alert responsive to the measured performance metric exceeding a threshold metric generated using the first environment descriptor for at least the first network environment.

4. The method of claim 1 , further comprising generating the first environment descriptor by combining an environment descriptor for the first network environment with an environment descriptor for a third network environment.

5. The method of claim 1 , further comprising grouping or categorizing the second network environment with the first network environment responsive to the first environment descriptor and the second environment descriptor being within a similarity threshold.

6. The method of claim 1 , comprising identifying the plurality of feature vectors using accesses to the application occurring within a time window.

7. The method of claim 1 , comprising providing the performance model by correlating the plurality of feature vectors identified from across the plurality of network environments, with the corresponding performance metrics.

8. The method of claim 1 , wherein generating the first environment descriptor comprises generating a distribution of the first expected performance metrics for at least the first network environment using the first expected performance metrics for at least the first network environment.

9. The method of claim 1 , wherein each of the plurality of feature vectors includes an application identifier referencing the application and at least one of a processor utilization, memory usage, or a network round trip time, describing the corresponding access, and
wherein the corresponding performance metric includes at least one of a launch time duration, an application response time, or a frequency of transactions in the corresponding access.

10. A system for generating environment descriptors, comprising:
at least one processor configured to:
identify a plurality of feature vectors each identifying a corresponding performance metric corresponding to access by one or more clients via at least one of a plurality of network environments to an application hosted on a server;
provide a performance model using the plurality of feature vectors and the corresponding performance metrics;
determine, using the performance model, first expected performance metrics corresponding to access of the application via at least a first network environment;
generate, based at least on the first expected performance metrics, a first environment descriptor for at least the first network environment to be used by the at least one first client to access the application; and
determine a similarity metric between the first environment descriptor of the first expected performance metrics and a second environment descriptor of second expected performance metrics for a second network environment to be used by at least one second client.

11. The system of claim 10 , wherein the at least one processor is further configured to generate, using the first environment descriptor of the first expected performance metrics, a threshold metric for identifying an outlier for at least the first network environment.

12. The system of claim 10 , wherein the at least one processor is further configured to generate an alert responsive to the measured performance metric exceeding a threshold metric generated using the first environment descriptor for at least the first network environment.

13. The system of claim 10 , wherein the at least one processor is further configured to generate the first environment descriptor by combining an environment descriptor for the first network environment with an environment descriptor for a third network environment.

14. The system of claim 10 , wherein the at least one processor is further configured to categorize the second network environment with the first network environment responsive to the first environment descriptor and the second environment descriptor being within a similarity threshold.

15. The system of claim 10 , wherein the at least one processor is further configured to provide the performance model by correlating the plurality of feature vectors identified from across the plurality of network environments, with the corresponding performance metrics.

16. The system of claim 10 , wherein the at least one processor is further configured to generate a distribution of the first expected performance metrics for at least the first network environment, using the first expected performance metrics for at least the first network environment.

17. A non-transitory computer readable medium storing instructions that when executed cause at least one processor to:
identify plurality of feature vectors each identifying a corresponding performance metric corresponding to access by one or more clients via at least one of a plurality of network environments to an application hosted on a server;
provide a performance model using the plurality of feature vectors and the corresponding performance metrics;
determine, using the performance model, first expected performance metrics corresponding to access of the application by at least one first client via at least a first network environment;
generate, based at least on the first expected performance metrics, a first environment descriptor for at least the first network environment to be used by the at least one first client to access the application; and
determine a similarity metric between the first environment descriptor of the first expected performance metrics and a second environment descriptor of second expected performance metrics for a second network environment to be used by at least one second client.

18. The non-transitory computer readable medium of claim 17 , wherein the instructions cause the at least one processor to categorize the second network environment with the first network environment responsive to the first environment descriptor and the second environment descriptor being within a similarity threshold.

Assignees

• Citrix Systems Inc

Inventors

• Varnavas Andreas
• Kumar Kiran
• Vellingiri Venkatesh

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F18/211

  Selection of the most significant subset of features · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L41/5009 Primary

  Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF] · CPC title