Method and system for monitoring and analyzing quality of service in a storage system

Patent metadata

Field	Value
Publication number	US-9547445-B2
Application number	US-201414535565-A
Country	US
Kind code	B2
Filing date	Nov 7, 2014
Priority date	Jan 14, 2014
Publication date	Jan 17, 2017
Grant date	Jan 17, 2017

Abstract

Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data based on the collected QOS data is generated. The process then determines a deviation of each potential bully storage volume of a resource used by any victim storage volume, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS and a predicted upper threshold TOPS value for each potential bully storage volume; and filters the potential bully storage volumes based on an impact of each potential bully storage volume.

First claim

What is claimed is:

1. A machine implemented method, comprising:
collecting quality of service (QOS) data for a plurality of storage volumes from a storage operating system of a storage system, each storage volume associated with a QOS policy; wherein the QOS data includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request;
generating an expected range for future QOS data based on the collected QOS data, where the expected range predicts future response times for the plurality of storage volumes and is a range of measured performance activity of a workload over a period of time;
monitoring QOS data for each storage volume for determining whether a current QOS data for each storage volume is within the expected range;
identifying a plurality of potential victim storage volumes when a response time of the storage volumes has reached a predefined, static threshold value and a dynamic threshold value that is based on the expected range;
filtering the plurality of potential victim storage volumes to determine victim storage volumes based on a deviation of each potential victim storage volume's response time, where the deviation is based on a current response time for each potential victim storage volume, a predicted response time that is based on the expected range and a predicted upper threshold response time for each potential victim storage volume that changes dynamically as more QOS data is collected for the plurality of storage volumes;
identifying a plurality potential bully storage volumes based on a visit rate, a service time and utilization of a resource that is in contention and used by any victim storage volume;
determining a deviation of each potential bully storage volume of the resource in contention, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS based on the collected data and a predicted upper threshold TOPS value for each potential bully storage volume; and
filtering the potential bully storage volumes based on an impact of each potential bully storage volume, where the impact of each potential bully storage volume depends on the deviation and the current number of IOPs for each bully storage volume.

2. The method of claim 1 , wherein the expected range provides the dynamic threshold value to a processor executable performance manager for predicting an incident associated with any of the plurality of storage volumes.

3. The method of claim 1 , wherein the expected range provides the dynamic threshold value to a processor executable performance manager for determining resource contention between any of the plurality of storage volumes resulting in a storage volume becoming a victim due to overuse by a bully storage volume.

4. The method of claim 1 , wherein the QOS data includes a wait time for each I/O request at each of the resources of the storage system.

5. The method of claim 1 , wherein the QOS data includes a number of visits for each I/O request at each of the resources within the storage system.

6. The method of claim 1 , wherein the resources at the storage system are categorized as a service center that services a request and a delay center that only moves the request to a next destination.

7. The method of claim 6 , wherein the resources include a processor for a network module that interfaces with clients and operates as a service center and a processor for a storage module that operates as a service center and manages storage devices where information is stored.

8. A non-transitory, machine readable storage medium having stored thereon instructions for performing a method, comprising machine executable code which when executed by at least one machine, causes the machine to:
collect quality of service (QOS) data for a plurality of storage volumes from a storage operating system of a storage system, each storage volume associated with a QOS policy; wherein the QOS data includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request;
generate an expected range for future QOS data based on the collected QOS data, where the expected range predicts future response times for the plurality of storage volumes and is a range of measured performance activity of a workload over a period of time;
monitor QOS data for each storage volume for determining whether a current QOS data for each storage volume is within the expected range;
identify a plurality of potential victim storage volumes when a response time of the storage volumes has reached a predefined, static threshold value and a dynamic threshold value that is based on the expected range;
filter the plurality of potential victim storage volumes to determine victim storage volumes based on a deviation of each potential victim storage volume's response time, where the deviation is based on a current response time for each potential victim storage volume, a predicted response time that is based on the expected range and a predicted upper threshold response time for each potential victim storage volume that changes dynamically as more QOS data is collected for the plurality of storage volumes;
identify a plurality potential bully storage volumes based on a visit rate, a service time and utilization of a resource that is in contention and used by any victim storage volume;
determine a deviation of each potential bully storage volume of the resource in contention, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS based on the collected data and a predicted upper threshold TOPS value for each potential bully storage volume; and
filter the potential bully storage volumes based on an impact of each potential bully storage volume, where the impact of each potential bully storage volume depends on the deviation and the current number of IOPs for each bully storage volume.

9. The storage medium of claim 8 , wherein the expected range provides the dynamic threshold value to a processor executable performance manager for predicting an incident associated with any of the plurality of storage volumes.

10. The storage medium of claim 8 , wherein the expected range provides the dynamic threshold value to a processor executable performance manager for determining resource contention between any of the plurality of storage volumes resulting in a storage volume becoming a victim due to overuse by a bully storage volume.

11. The storage medium of claim 8 , wherein the QOS data includes a wait time for each I/O request at each of the resources of the storage system.

12. The storage medium of claim 8 , wherein the QOS data includes a number of visits for each I/O request at each of the resources within the storage system.

13. The storage medium of claim 8 , wherein the resources at the storage system are categorized as a service center that services a request and a delay center that only moves the request to a next destination.

14. The storage medium of claim 13 , wherein the resources include a processor for a network module that interfaces with clients and operates as a service center and a processor for a storage module that operates as a service center and manages storage devices where information is stored.

15. A system, comprising:
a memory containing machine readable medium comprising machine executable code having stored thereon instructions; and a processor module coupled to the mem

Assignees

• Netapp Inc

Inventors

• Faulkner Kevin
• Fairbanks Leon
• Wu Siu
• Pai Vinod Vasant
• Hrischuk Curtis

Classifications

• G06F3/0611 Primary

  in relation to response time · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title

• G06F2003/0692

  Physics · mapped topic

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title