Self adaptive workload classification and forecasting in multi-tiered storage system using ARIMA time series modeling

Patent metadata

Field	Value
Publication number	US-9703664-B1
Application number	US-201514748709-A
Country	US
Kind code	B1
Filing date	Jun 24, 2015
Priority date	Jun 24, 2015
Publication date	Jul 11, 2017
Grant date	Jul 11, 2017

Abstract

Techniques are described data storage optimization that determine predicted values for I/O statistics using an ARIMA (auto-regressive integrated moving average) model. The ARIMA model may be used to capture periodic patterns and trends of workload I/O access to predict the future load demand. A current set of I/O statistics is collected for a current time period T. Using the current set and one or more ARIMA models, a predicted set of I/O statistics is determined for a next time period T+1. Each of the ARIMA models is characterized by model parameters including P denoting a number of auto-regressive terms, D denoting a number of nonseasonal difference needed for stationarity, and Q denoting a number of lagged forecast errors of prediction. A data storage optimizer may determine one or more data portions for movement from a current storage tier to a target storage tier using the predicted set of I/O statistics.

First claim

What is claimed is:

1. A method of performing data storage optimization comprising:
collecting a current set of one or more I/O statistics for a current time period T;
determining, using the current set of one or more I/O statistics and one or more models, a predicted set of one or more I/O statistics for a next time period T+1, wherein each of the one or more models is used to model at least a first of the one or more I/O statistics of the predicted set, wherein each of the one or more models is an auto-regressive integrate moving average model characterized by model parameters including P denoting a number of auto-regressive terms, D denoting a number of nonseasonal difference needed for stationarity, and Q denoting a number of lagged forecast errors of prediction; and
determining, by a data storage optimizer using the predicted set of one or more I/O statistics, one or more data portions for movement from a current storage tier to a target storage tier, wherein each of the one or more I/O statistics of the predicted set is a predicted I/O statistic for the next time period T+1 determined using a constant and any of: a weighted sum of one or more observed values for said each I/O statistic in one or more time periods prior to time period T+1, and a weighted sum of one or more error values each denoting an amount of error between a predicted value and an observed value for said each I/O statistic in a particular time period, wherein at least a first of the one or more data portions selected for movement is included in a logical device having an average observed response time that is not in accordance with a specified response time objective denoting a performance goal for the logical device, and wherein the logical device has the average observed response time that is greater than the specified response time objective and the method includes:
moving the first data portion from the current storage tier to the target storage tier, wherein the current storage tier has an associated expected response time that is more than the specified response time objective for the logical device and wherein said target storage tier is a higher performance storage tier than the current storage tier.

2. The method of claim 1 , further comprising:
determining, by the data storage optimizer using the predicted set of one or more I/O statistics, a second data portion for movement, wherein the second data portion is included in a first storage pool of storage devices of a first storage tier and the first storage pool has an associated average observed response time that is more than a second specified response time objective denoting an expected service level objective for physical storage devices of the first storage tier.

3. The method of claim 2 , further comprising:
moving the second data portion from the first storage tier to another storage tier that is a higher performance storage tier than the first storage tier.

4. The method of claim 1 , further comprising:
determining, by the data storage optimizer using the predicted set of one or more I/O statistics, a second data portion for movement, wherein the second data portion is included in a first storage pool of storage devices of a first storage tier and the first storage pool has a maximum capacity limit denoting an upper allocation bound wherein a current allocation amount of the first storage pool exceeds the maximum capacity limit.

5. The method of claim 4 , wherein the second data portion is included in a first extent determined to be idle or has a predicted I/O workload that is lower than any other extent including any of the one or more data portions.

6. The method of claim 1 , wherein the current set of one or more I/O statistics for the current time period T includes any of a random read miss I/O statistic, a write I/O statistic, and a read miss sequential I/O statistic.

7. The method of claim 1 , wherein the predicted set of one or more I/O statistics for a next time period T+1 includes one or more predicted short term I/O statistics and one or more predicted long term I/O statistics.

8. The method of claim 7 , wherein the one or more predicted short term I/O statistics includes a predicted short term read I/O statistic and a predicted short term write I/O statistic.

9. The method of claim 7 , wherein the one or more predicted long term I/O statistics includes a predicted long term read I/O statistic and a predicted long term write I/O statistic.

10. The method of claim 1 , wherein the logical device is a virtually provisioned logical device.

11. A method of performing data storage optimization comprising:
collecting a current set of one or more I/O statistics for a current time period T;
determining, using the current set of one or more I/O statistics and one or more models, a predicted set of one or more I/O statistics for a next time period T+1, wherein each of the one or more models is used to model at least a first of the one or more I/O statistics of the predicted set, wherein each of the one or more models is an auto-regressive integrate moving average model characterized by model parameters including P denoting a number of auto-regressive terms, D denoting a number of nonseasonal difference needed for stationarity, and Q denoting a number of lagged forecast errors of prediction; and
determining, by a data storage optimizer using the predicted set of one or more I/O statistics, one or more data portions for movement from a current storage tier to a target storage tier, wherein each of the one or more models uses an equation or function including a constant, a first set of one or more terms that are the auto-regressive terms and a second set of one or more terms that are lagged forecast errors of prediction, wherein at least a first of the one or more models uses a first set of one or more coefficients and a second set of one or more coefficients, said first set of coefficients being used with the first set of one or more terms and said second set of coefficients being used with the second set of one or more terms, wherein at least a first of the one or more data portions selected for movement is included in a logical device having an average observed response time that is not in accordance with a specified response time objective denoting a performance goal for the logical device, and wherein the logical device has the average observed response time that is greater than the specified response time objective and the method includes:
moving the first data portion from the current storage tier to the target storage tier, wherein the current storage tier has an associated expected response time that is more than the specified response time objective for the logical device and wherein said target storage tier is a higher performance storage tier than the current storage tier.

12. The method of claim 11 , wherein any of the first set of one or more coefficients, the second set of one or more coefficients and a value for parameter D is determined in accordance with I/O workload characteristics of an application issuing I/O operations directed to a set of one or more logical devices including the one or more data portions selected for data movement, wherein said collecting collects information regarding I/O operations of the application observed during the current time period T.

13. The method of claim 12 , wherein the I/O workload characteristics denote any of: whether I/O workload for the application includes more reads than writes, whether I/O workload for the application includes more sequential I/Os than random I/Os, whether I/O workload for the application includes I/Os larger than a specified size, and whether

Assignees

• Emc Ip Holding Co Llc

Inventors

• Alshawabkeh Malak
• Martin Owen

Classifications

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title

• G06F11/3414 Primary

  Workload generation, e.g. scripts, playback · CPC title

• G06F3/0649

  Lifecycle management · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title