Method and system that analyzes operational characteristics of multi-tier applications

Patent metadata

Field	Value
Publication number	US-2016337226-A1
Application number	US-201514711648-A
Country	US
Kind code	A1
Filing date	May 13, 2015
Priority date	May 13, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

The current document is directed to an analysis subsystem within a large distributed computing system, such as a virtual data center or cloud-computing facility, that monitors the operational states associated with a multi-tiered application and provides useful information for determining one or more causes of various types of failures and undesirable operational states that may arise during operation of the multi-tiered application. In one implementation, the analysis subsystem collects metrics provided by various different types of metrics sources within the computational system and employs principal feature analysis to select a generally small subset of the collected metrics particularly relevant to monitoring a multi-tiered application and diagnosing underlying causes of operational states of the multi-tiered application. The analysis subsystem develops one or more conditional probability distributions with respect to the subset of metrics. These one or more conditional probability distributions, in turn, allow the analysis subsystem to provide useful information for analysis of the causes of failures and undesirable system states associated with the multi-tiered application.

First claim

1 . An analysis subsystem that monitors a multi-tiered application within a distributed computing system, the analysis subsystem comprising:
one or more processors; one or more memories; one or more mass-storage devices; communications subsystems that allow the analysis subsystem to communicate with external servers and other computational entities; metric-data storage containers, implemented within one or more of the one or more memories and one or more mass-storage devices, that store metric data; and computer instructions, stored in one or more of the one or more memories that, when executed by the one or more processors, control the analysis subsystem to
collect infrastructure metrics from one or more infrastructure metrics sources in one or more metric-data storage containers,
collect system and application metrics from one or more system and application metrics sources in one or more metric-data storage containers,
process the collected infrastructure, system, and application metrics to select a subset of the infrastructure, system, and application metrics relevant to the multi-tiered application that are then stored in one or more metric-data storage containers,
use the selected subset of the infrastructure, system, and application metrics to generate a representation of a conditional probability distribution for the selected subset of the infrastructure, system, and application metrics, and
use the representation of the conditional probability distribution for the selected subset of the infrastructure, system, and application metrics to generate data for transmission to, and storage within, one or more of diagnostics, management, and monitoring subsystems.

2 . The analysis subsystem of claim 1 wherein infrastructure metrics include one or more of:
metrics that represent mappings of virtual machines to servers within the distributed computing system;
metrics that represent virtual-machine utilizations of distributed-computing-system storage resources; and
metrics that represent virtual-machine utilizations of distributed-computing-system network resources

3 . The analysis subsystem of claim 1 wherein system metrics include one or more of:
metrics that represent CPU utilization;
metrics that represent memory utilization; and
metrics that represent storage utilization.

4 . The analysis subsystem of claim 1 wherein application metrics include one or more of:
metrics that represent latency in application responses to requests;
metrics that represent throughput of requests to the application; and
metrics that represent that number of accesses to resources per unit period of time.

5 . The analysis subsystem of claim 1 wherein the infrastructure metrics sources include:
an operations monitor running within an operations server; and
an operations monitor running within a distributed-computing-system management server.

6 . The analysis subsystem of claim 1 wherein system and application metrics sources include:
a Hyperic server that collects system and application metrics from Hyperic agents within application servers;
a system monitor running within a management server; and
an application monitor running within a management server.

7 . The analysis subsystem of claim 1 wherein the analysis subsystem processes the collected infrastructure, system, and application metrics to select a subset of the infrastructure, system, and application metrics relevant to the multi-tiered application by carrying out principle feature analysis.

8 . The analysis subsystem of claim 7 wherein principle feature analysis comprises:
collecting n metric row vectors X 1 , X 2 , . . . , X n , each of dimension p, into a matrix X;
constructing a column vector u with p entries, each entry of vector u including a numeric mean of the entries in a column of the matrix X;
transposing the column vector u to a row vector and multiplying the row vector with a column vector containing all “1” entries to produce a matrix M;
subtracting the matrix M from the matrix X to produce s matrix B containing mean-subtracted metric data;
multiplying the matrix B with the transpose of matrix B, B T , and multiplying the resulting p×p matrix by 1/n−1 to produce a covariance matrix C;
determining eigenvalues λ 1 , λ 2 , . . . , λ p and eigenvectors v 1 , v 2 , . . . , v p of the covariance matrix C;
combining the eigenvectors together to produce a matrix V;
using the matrix V to diagonalize the covariance matrix C to produce the diagonal matrix D having values along the diagonal that are the eigenvalues of the covariance matrix;
rearranging the eigenvalues in order of decreasing magnitude, with the corresponding eigenvectors identically rearranged, to produce matrices D′ and V′;
using the matrix D′ to choose a value q as the dimension of the subset of the metric data generated by principal feature analysis;
selecting the first q columns of matrix V′ to form the p×q matrix A q ;
selecting p q-dimensional rows v 1 , v 2 , . . . , v p from the matrix A q ;
using k-means clustering to cluster the selected rows of matrix A q into r clusters, where p>r>q; and
selecting the q-dimensional vectors closest to the centroids of each cluster as the features for lower-dimensional metric vectors that together form the metric-data subset generated by the principal feature analysis.

9 . The analysis subsystem of claim 8 wherein using the matrix D′ to choose a value q as the dimension of the subset of the metric data generated by principal feature analysis further comprises:
defining a value g j as the sum of the values along the diagonal of matrix D′ down to and including the j th eigenvalue λ′ j ; and
choosing a value q so that g q is greater than or equal to a threshold percentage times g p .

10 . The analysis subsystem of claim 8 wherein k-means clustering comprises:
selecting r vectors from the p q-dimensional vectors v 1 , v 2 , . . . , v p selected from the matrix A q as representative vectors for r clusters that are used as the initial centroids for the clusters; and
iteratively reconstructing the r clusters until there is no change in cluster membership from the most recently constructed set of clusters and the previous set of clusters by
assigning each unassigned vector to a cluster with a centroid closest to the unassigned vector, and
computing a new centroid for each cluster in step.

11 . The analysis subsystem of claim 1 wherein the representation of a conditional probability distribution for the selected subset of the infrastructure, system, and application metrics comprises one or more Bayesian networks selected from among:
Bayesian networks,
naive Bayesian networks;
tree-augmented naive Bayesian networks; and
Bayesian, naive Bayesian, or tree-augmented naive Bayesian networks augmented with additional information.

12 . The analysis subsystem of claim 11 wherein a tree-augmented naive Bayesian network includes:
a first node representing a special independent metric;
additional nodes each representing a metric that is conditionally dependent on the first node and conditionally dependent on at most one other of the additional nodes; and
directed edges representing conditional dependencies.

13 . The analysis subsystem of claim 13 wherein a tree-augmented naive Bayesian network is constructed from the selected subset of the infrastructure, system, and application metrics relevant to the multi-tiered application, represented as a set of metric vectors, by:
generating a mutual conditional information matrix from the metric vectors; 1802 . As indicated by diagonal line 1806 , the mutual-conditional-inf

Assignees

• Vmware Inc

Inventors

• Padala Pradeep
• Mukiri Neelima
• Marvasti Mazda A

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• H04L43/10 Primary

  Active monitoring, e.g. heartbeat, ping or trace-route · CPC title

• G06F9/45558

  Hypervisor-specific management and integration aspects · CPC title

• G06N7/005

  Physics · mapped topic

• G06F2009/45595

  Network integration; Enabling network access in virtual machine instances · CPC title