System and method for adaptive data transfers with limited resources

Patent metadata

Field	Value
Publication number	US-2016196089-A1
Application number	US-201514591557-A
Country	US
Kind code	A1
Filing date	Jan 7, 2015
Priority date	Jan 7, 2015
Publication date	Jul 7, 2016
Grant date	—

Abstract

A method and system for adaptive data transfers with limited resources are described. A data storage system performs operations that include migrating data from one set of storage volumes to another set of storage volumes using a number of data streams managed by a storage controller. During this data migration process, the storage controller can measure performance of the data storage system such as the CPU load, memory usage, network bandwidth usage, and the total number of data streams in use. Based on the measured performance and scheduling information, the data storage system can adjust the number of concurrent data streams dedicated to data migration in order to complete the migration process in a timely fashion without substantial undue impact to client accesses and backup operations.

First claim

What is claimed is:

1 . A method of data migration, the method being implemented by one or more processors and comprising:
migrating data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller; measuring performance of the storage controller during data migration; and adjusting the number of concurrent data streams used for migrating data based on the measured performance.

2 . The method of claim 1 , wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

3 . The method of claim 1 , wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

4 . The method of claim 1 , wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

5 . The method of claim 4 , wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

6 . The method of claim 1 , further comprising:
analyzing a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and reducing the number of concurrent data streams migrating data during times corresponding to the schedule.

7 . The method of claim 6 , wherein the data streams determined to be more critical than the number of concurrent data streams migrating data are streams that backup data from the first set of storage volumes.

8 . A data migration system comprising:
a memory resource to store instructions; one or more processors using the instructions stored in the memory resource to: migrate data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller; measure performance of the storage controller during data migration; and adjust the number of concurrent data streams used for migrating data based on the measured performance.

9 . The system of claim 8 , wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

10 . The system of claim 8 , wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

11 . The system of claim 8 , wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

12 . The system of claim 11 , wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

13 . The system of claim 8 , further comprising instructions to:
analyze a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and reduce the number of concurrent data streams migrating data during times corresponding to the schedule.

14 . The system of claim 13 , wherein the data streams determined to be more critical than the number of concurrent data streams migrating data are streams that backup data from the first set of storage volumes.

15 . A non-transitory computer-readable medium for implementing data migration, the non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations that include:
migrating data from a first set of storage volumes to a second set of storage volumes, wherein migrating data uses a number of concurrent data streams managed by a storage controller; measuring performance of the storage controller during data migration; and adjusting the number of concurrent data streams used for migrating data based on the measured performance.

16 . The non-transitory computer-readable medium of claim 15 , wherein the first set of storage volumes is associated with a first architecture, and the second set of storage volumes is associated with a second architecture that is different from the first architecture.

17 . The non-transitory computer-readable medium of claim 15 , wherein measuring performance of the storage controller includes measuring CPU load, memory usage, network bandwidth usage, number of critical and non-critical data streams in use, and number of available data streams.

18 . The non-transitory computer-readable medium of claim 15 , wherein the number of concurrent data streams used for migrating data is increased when the performance of the storage controller is above a first programmed threshold, and wherein the number of concurrent data streams used for migrating data is decreased when the performance of the storage controller is below a second programmed threshold.

19 . The non-transitory computer-readable medium of claim 18 , wherein the first and second programmed thresholds are set by an individual with access to the storage controller.

20 . The non-transitory computer-readable medium of claim 15 , further comprising:
analyzing a schedule of data streams determined to be more critical than the number of concurrent data streams migrating data; and reducing the number of concurrent data streams migrating data during times corresponding to the schedule.

Assignees

• Netapp Inc

Inventors

• Gadre Vijay Vinayak
• Reddy Ch Vijaya Bhaskar
• Revsin Vitaly

Classifications

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F3/0665

  at area level, e.g. provisioning of virtual or logical volumes · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0647 Primary

  Migration mechanisms · CPC title