Adaptive learning techniques for determining expected service levels

Patent metadata

Field	Value
Publication number	US-9946465-B1
Application number	US-201414587106-A
Country	US
Kind code	B1
Filing date	Dec 31, 2014
Priority date	Dec 31, 2014
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

Described are techniques user to determine expected service levels. A first set of one or more data portions are selected from one or more devices forming a first device set. The first set of data portions are stored on physical storage of a physical device set of one or more physical devices having an unknown service level expectation. I/O operations directed to the first set of data portions are received and serviced for a time period. First information is collected that characterizes performance of the physical device set during the time period. In accordance with the first information and using an adaptive learning technique, a first service level expectation for servicing I/Os directed to the physical device set is determined.

First claim

What is claimed is:

1. A method of determining expected service levels comprising:
selecting, using a processor, a first set of data portions from one or more logical devices, said one or more logical devices forming a logical device set;
storing, using a processor, said first set of data portions on physical storage of a physical device set of one or more physical devices having an unknown service level expectation with respect to servicing I/O operations directed to the physical device set;
receiving and processing, using a processor, I/O operations directed to said first set of data portions for a time period;
collecting, using a processor, first information characterizing performance of the physical device set during said time period; and
determining, in accordance with the first information and using a processor and an adaptive learning technique, a first service level expectation for servicing I/Os directed to the physical device set, wherein said determining the first service level expectation includes:
determining, using the first information, an overall cumulative distribution function including a plurality of response time bins;
determining a first of the plurality of response time bins meeting threshold criteria, said first response time bin having a first response time denoting a maximum response time of the first response time bin; and
using said first response time as said first service level expectation denoting an expected response time for I/Os directed to the physical device set.

2. The method of claim 1 , wherein each of a plurality of storage tiers has an associated one of a plurality of service level expectations denoting an expected performance for I/O operations directed to a physical device of said each storage tier, said plurality of storage tiers including the physical device set as a first of said plurality of storage tiers, and wherein said first service level expectation is one of the plurality of service level expectations and is associated with said first storage tier specifying an expected performance for I/O operations directed to the first storage tier.

3. The method of claim 2 , further comprising:
receiving a first service level objective specifying a performance goal for a first logical device of the logical device set;
collecting first I/O workload information for a plurality of data portions of the first logical device stored on one or more of the plurality of storage tiers; and
determining, in accordance with the first service level objective and said first I/O workload information, a first partitioning of the plurality of data portions among the plurality of storage tiers.

4. The method of claim 3 , wherein said determining said first partitioning includes modeling performance for I/O operations directed to the first logical device using the first I/O workload information and said plurality of service level expectations for said plurality of storage tiers, wherein said modeling determines that said first partitioning results in an estimated performance for I/O operations directed to the first logical device wherein the estimated performance meets said performance goal specified by the first service level objective for the first logical device.

5. The method of claim 3 , wherein each of the plurality of data portions is stored on a physical device of one of the plurality of storage tiers selected in accordance with an I/O workload directed to said each data portion.

6. The method of claim 3 , wherein the logical device is virtually provisioned.

7. The method of claim 1 , wherein said first service level expectation is a response time denoting an expected response time for I/O operations directed to a data portion stored on a physical device of said physical device set.

8. The method of claim 1 , wherein each data portion of the first set has a lower I/O workload than any data portion of the one or more devices not in the first set of data portions.

9. The method of claim 1 , further comprising:
receiving and processing a second set of I/O operations directed to data portions stored on the physical device set for a second time period;
collecting second information characterizing performance of the physical device set during said second time period; and
revising, in accordance with the second information and said adaptive learning technique, said first service level expectation for servicing I/Os directed to the physical device set.

10. The method of claim 9 , wherein said time period occurs at a first point in time and said second time period occurs at a second point in time subsequent to the first point in time, and wherein said revising said first service level expectation for servicing I/Os directed to the physical device set updates said first level expectation from a first value determined for the time period to a second value determined in accordance with the second information, said second value reflecting changes in the set of one or more physical devices of the physical device set between said first point in time and said second point in time.

11. The method of claim 10 , wherein the physical device set includes one or more storage devices subject to device wear out based on any of write operations and erase operations, and said second value reflecting a change in performance due to said device wear out since said first point in time.

12. The method of claim 11 , wherein the one or more storage devices are flash-based storage devices.

13. The method of claim 10 , wherein the physical device set includes one or more storage devices that are rotating disk drives and said second value reflects a change in performance due to a temperature increase or decrease since said first point in time.

14. The method of claim 1 , further comprising:
collecting, using a processor, a plurality of data sets for a plurality of time periods, wherein each of the plurality of data sets is collected during one of the plurality of time periods and said each data set includes a set of values for a plurality of parameters characterizing I/O workload for said one time period and including a response time histogram characterizing response time for said one time period;
determining, using a processor, one of a plurality of I/O workload classifications for each of the plurality of data sets in accordance with said set of values of said each data set; and
for each of the plurality of I/O workload classifications including more than one of the plurality of data sets, combining, using a processor, said more than one of the plurality of data sets into a first aggregate data set including an aggregate set of values in accordance with said set of values of each of said more than one data set and including an aggregate response time histogram in accordance with said response time histogram of each of said more than one data set.

15. The method of claim 14 , further comprising:
determining, using a processor, a plurality of cumulative distribution functions for said plurality of I/O workload classifications;
determining, in accordance with said plurality of cumulative distribution functions and using a processor, said overall cumulative distribution function, wherein said overall cumulative distribution function indicates, for each of said plurality of response time bins, a percentage of I/O operations having a response time less than a maximum response time of said each response time bin; and
determining, using and processor and for each of the plurality of response time bins of said overall cumulative distribution function, an average percentage and any of: a standard deviation with respect t

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Martin Owen
• Alshawabkeh Malak
• Randolph Benjamin

Classifications

• G06F3/061 Primary

  Improving I/O performance · CPC title

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G05B13/0265

  the criterion being a learning criterion · CPC title

• G06F3/0664

  at device level, e.g. emulation of a storage device or system · CPC title

• G06F3/0688

  Non-volatile semiconductor memory arrays · CPC title