Modeling storage system performance

Patent metadata

Field	Value
Publication number	US-9514022-B1
Application number	US-201113275607-A
Country	US
Kind code	B1
Filing date	Oct 18, 2011
Priority date	Oct 18, 2011
Publication date	Dec 6, 2016
Grant date	Dec 6, 2016

Abstract

A system and method for creating an accurate black-box model of a live storage system and for predicting performance of the storage system under a given workload is disclosed. An analytics engine determines a subset of counters that are relevant to performance of the storage system with respect to a particular output (e.g., throughput or latency) from performance data in counters of the storage system. Using the subset of counters, the analytics engine creates a workload signature for the storage system by using a recursive partitioning technique, such as a classification and regression tree. The analytics engine then creates the black-box model of the storage system performance by applying uncertainty measurement techniques, such as a Gaussian process, to the workload signature.

First claim

What is claimed is:

1. A method performed by a storage system having one or more processors, comprising:
collecting storage system performance data using a plurality of counters;
filtering by an analytics engine of the storage system the collected storage system performance data by eliminating any counter value that is constant over time and any counter that has a zero value, wherein the one or more processors invoke the analytics engine and the filtering generates a first subset of counters with associated performance data that affects performance of the storage system relative to a service level objective;
identifying by the analytics engine which of the first subset of counters are monotonic with respect to a performance parameter that is used for predicting performance of the storage system relative to the service level objective;
selecting by the analytics engine a second subset of counters from the first subset of counters based on a degree of monotonicity where the degree of monotonicity selects lower deviation monotonic counters over higher deviation monotonic counters from among the first subset of counters;
determining by the analytics engine a third subset of counters from the second subset of counters by using an initial regression model with a first threshold value and a second regression model with a second threshold value that is less than the first threshold value to determine whether the third subset of counters are a reliable indicator of performance of the storage system relative to the service level objective;
using by the analytics engine recursive partitioning for iteratively determining regression error of the second regression model for the third subset of counters to identify a fourth subset of counters with minimum regression error;
creating a workload signature of the storage system relative to the service level objective and by using recursive partitioning of performance data associated with the fourth subset of counters and representing a plurality of measures of actual live storage system performance, wherein the workload signature is based on the fourth subset of counters and the minimum regression error associated with the fourth subset of counters;
creating, by the analytics engine, a storage system performance model by using the workload signature and an uncertainty measurement technique, wherein the uncertainty measurement technique applies a Gaussian process model to each leaf node of the recursive partitioning and computes a confidence band indicating a certain confidence level for a likelihood of actual performance to be within a specified deviation from a prediction of storage system performance; and
predicting, by the analytics engine, performance of the storage system under a sample load by applying the sample load to the storage system performance model.

2. The method of claim 1 , wherein the recursive partitioning comprises a classification and regression tree.

3. The method of claim 1 further comprising:
comparing, by the analytics engine, the predicted performance to actual performance of the live storage system; and
in response to determining that the actual performance is not consistent with the predicted performance, repeating the steps of creating a workload signature, creating a storage system performance model, and predicting performance of the live storage system.

4. The method of claim 1 , further comprising generating a graphical user interface that depicts the predicted performance of the storage system and the actual performance of the storage system, wherein the graphical interface further provides an indication of one or more regions where the actual performance is not consistent with the predicted performance.

5. The method of claim 1 , further comprising generating a graphical user interface that depicts the workload signature of the storage system and one or more past workload signatures of the storage system.

6. The method of claim 1 , further comprising:
determining, based on the predicted performance of the storage system, an incremental provisioning of a workload or a workload distribution to meet one or more service level objectives; and
implementing the determined incremental provisioning or workload distribution to meet the one or more service level objectives.

7. The method of claim 1 , wherein the service level objective is one of latency or throughput.

8. The method of claim 1 , wherein the analytics engine constructs a variogram that describes a degree of correlation between two observations to determine the monotonicity of the first subset of counters.

9. A system comprising:
a memory containing machine readable medium comprising machine executable code having stored thereon instructions; and a processor module coupled to the memory executes the machine executable code for an analytics engine to:
collect performance data for a storage system using a plurality of counters;
filter the collected storage system performance data by eliminating any counter value that is constant over time and any counter that has a zero value, wherein the filtering generates a first subset of counters with associated performance data that affects performance of the storage system relative to a service level objective;
identify which of the first subset of counters are monotonic with respect to a performance parameter that is used for predicting performance of the storage system relative to the service level objective;
select a second subset of counters from the first subset of counters based on a degree of monotonicity where the degree of monotonicity selects lower deviation monotonic counters over higher deviation monotonic counters from among the first subset of counters;
determine a third subset of counters from the second subset of counters by using an initial regression model with a first threshold value and a second regression model with a second threshold value that is less than the first threshold value to determine whether the third subset of counters are a reliable indicator of performance of the storage system relative to the service level objective;
use recursive partitioning for iteratively determining regression error of the second regression model for the third subset of counters to identify a fourth subset of counters with minimum regression error;
create a workload signature of the storage system relative to the service level objective and by using recursive partitioning of performance data associated with the fourth subset of counters and representing a plurality of measures of actual live storage system performance, wherein the workload signature is based on the fourth subset of counters and the minimum regression error associated with the fourth subset of counters;
create a storage system performance model by using the workload signature and an uncertainty measurement technique, wherein the uncertainty measurement technique applies a Gaussian process model to each leaf node of the recursive partitioning and computes a confidence band indicating a certain confidence level for a likelihood of actual performance to be within a specified deviation from a prediction of storage system performance; and
predict performance of the storage system under a sample load by applying the sample load to the storage system performance model.

10. The system of claim 9 , wherein a graphical user interface depicts the predicted performance of the storage system and the actual performance of the storage system and provides an indication of one or more regions where the actual performance is not consistent with the predicted performance.

11. The system of claim 10 , wherein the graphical user interface depicts the workload signature

Assignees

• Basak Jayanta
• Voruganti Kaladhar
• Nandi Siddhartha
• Netapp Inc

Inventors

• Basak Jayanta
• Voruganti Kaladhar
• Nandi Siddhartha

Classifications

• G06F11/00

  Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00) · CPC title

• G06F11/3452 Primary

  Performance evaluation by statistical analysis · CPC title

• G06F11/3447 Primary

  Performance evaluation by modeling · CPC title

• G06F11/3034

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/3485

  for I/O devices · CPC title