System and method for smart throttling mechanisms for virtual backup appliances

Patent metadata

Field	Value
Publication number	US-9628561-B1
Application number	US-201414578310-A
Country	US
Kind code	B1
Filing date	Dec 19, 2014
Priority date	Dec 19, 2014
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

A method for managing resources of virtual backup appliances is described. The method includes receiving, by a backup management server (BMS), a request to back up a plurality of virtual machines (VMs), determining, at the BMS, a set of VMs to be backed up from the plurality of VMs that yields a maximum performance for the backup request without causing the set of VM servers to become bottlenecked, wherein the determination is performed based on at least one of a concurrency limit of the virtual backup appliance (VBA) and any VBA proxies corresponding to the set of VMs, and a set of one or more data performance metrics of one or more storage devices in the set of VM storage arrays, and sending one or more backup jobs to the VBA to back up the determined set of VMs to be backed up to a backup storage system.

First claim

What is claimed is:

1. A method for managing resources of virtual backup appliances, the method comprising:
receiving, by a backup management server (BMS), a request to back up a plurality of virtual machines (VMs) of a set of one or more VM servers, wherein each VM includes one or more VM disk files that are stored on a set of one or more VM storage arrays, wherein the BMS is configured to manage backup operations to back up data from the set of VM storage arrays to a backup storage system over a network, wherein a primary VM server of the set of VM servers has a virtual backup appliance (VBA), wherein any remaining VM servers of the set of VM servers each have a VBA proxy, and wherein the VBA receives backup jobs from the BMS and communicates with any VBA proxies to carry out the backup jobs;
determining, at the BMS, a set of VMs to be backed up from the plurality of VMs in the backup request that yields a maximum performance for the backup request without causing the set of VM servers to become bottlenecked, wherein the determination is performed based on at least one of a concurrency limit of the VBA and any VBA proxies corresponding to the set of VMs, and a set of one or more data throughput performance metrics of one or more storage devices in the set of VM storage arrays, wherein the set of data throughput performance metrics includes a set of one or more maximum read performance throughput limits of each of the one or more storage devices in the set of VM storage arrays, and wherein the set of VMs to be backed up is determined from the plurality of VMs in the backup request such that a net aggregate maximum read performance throughput is reached for the set of VM storage arrays when the VBA and any VBA proxies carry out the back up of the set of VMs to be backed up, and wherein the concurrency limit of the VBA and any VBA proxies is the maximum number of VMs that each VBA and VBA proxy can process at once, and wherein the BMS further requests concurrency limits from the VBA and any VBA proxies prior to the sending of the one or more backup jobs, and wherein the set of VMs to back up is of a number equal to the net concurrency limit of the VBA and any VBA proxies; and
sending one or more backup jobs to the VBA to back up the determined set of VMs to be backed up to the backup storage system.

2. The method of claim 1 , wherein the net aggregate maximum read performance throughput is reached when the set of VMs to be backed up is selected to have VM disk files that when backed up by the VBA and any VBA proxies, generate the maximum read throughput limit in the most number of the one or more storage devices in the set of VM storage arrays.

3. The method of claim 1 , wherein the set of data throughput performance metrics includes a set of one or more latency measurements for the one or more storage devices in the set of VM storage arrays as measured by the BMS while the VBA and any VBA proxies carry out the one or more backup jobs, wherein, for each of the one or more storage devices, the BMS sends an increasing number of the one or more backup jobs to the VBA for VMs that have VM disk files on that storage device until the latency measurement for that storage device exceeds a predetermined threshold, and wherein the set of VMs to be backed up are those VMs that are associated with the one or more backup jobs.

4. The method of claim 3 , wherein the BMS adjusts an input/output (I/O) rate of all the backup jobs for VMs with VM disk files on a saturated storage device such that each job requests a similar amount of data transfer from that saturated storage device, wherein a saturated storage device is a storage device with a latency measurement that exceeds the predetermined threshold.

5. The method of claim 1 , wherein the BMS includes a backup policy store to store at least one of the concurrency limit of the VBA and any VBA proxies corresponding to the set of VMs, and the set of one or more data performance metrics.

6. A non-transitory machine readable storage medium comprising instructions for managing resources of virtual backup appliances, which when executed by a computing system causes the computing system to perform operations comprising:
receiving, by a backup management server (BMS), a request to back up a plurality of virtual machines (VMs) of a set of one or more VM servers, wherein each VM includes one or more VM disk files that are stored on a set of one or more VM storage arrays, wherein the BMS is configured to manage backup operations to back up data from the set of VM storage arrays to a backup storage system over a network, wherein a primary VM server of the set of VM servers has a virtual backup appliance (VBA), wherein any remaining VM servers of the set of VM servers each have a VBA proxy, and wherein the VBA receives backup jobs from the BMS and communicates with any VBA proxies to carry out the backup jobs;
determining, at the BMS, a set of VMs to be backed up from the plurality of VMs in the backup request that yields a maximum performance for the backup request without causing the set of VM servers to become bottlenecked, wherein the determination is performed based on at least one of a concurrency limit of the VBA and any VBA proxies corresponding to the set of VMs, and a set of one or more data throughput performance metrics of one or more storage devices in the set of VM storage arrays, wherein the set of data throughput performance metrics includes a set of one or more maximum read performance throughput limits of each of the one or more storage devices in the set of VM storage arrays, and wherein the set of VMs to be backed up is determined from the plurality of VMs in the backup request such that a net aggregate maximum read performance throughput is reached for the set of VM storage arrays when the VBA and any VBA proxies carry out the back up of the set of VMs to be backed up, and wherein the concurrency limit of the VBA and any VBA proxies is the maximum number of VMs that each VBA and VBA proxy can process at once, and wherein the BMS further requests concurrency limits from the VBA and any VBA proxies prior to the sending of the one or more backup jobs, and wherein the set of VMs to back up is of a number equal to the net concurrency limit of the VBA and any VBA proxies; and
sending one or more backup jobs to the VBA to back up the determined set of VMs to be backed up to the backup storage system.

7. The non-transitory computer readable storage medium of claim 6 , wherein the net aggregate maximum read performance throughput is reached when the set of VMs to be backed up is selected to have VM disk files that when backed up by the VBA and any VBA proxies, generate the maximum read throughput limit in the most number of the one or more storage devices in the set of VM storage arrays.

8. The non-transitory computer readable storage medium of claim 6 , wherein the set of data throughput performance metrics includes a set of one or more latency measurements for the one or more storage devices in the set of VM storage arrays as measured by the BMS while the VBA and any VBA proxies carry out the one or more backup jobs, wherein, for each of the one or more storage devices, the BMS sends an increasing number of the one or more backup jobs to the VBA for VMs that have VM disk files on that storage device until the latency measurement for that storage device exceeds a predetermined threshold, and wherein the set of VMs to be backed up are those VMs that are associated with the one or more backup jobs.

9. The non-transitory computer readable storage medium of claim 8 , wherein the BMS adjusts an input/output (I/O) rate of all the backup jobs for VMs with VM disk files on a saturated storage device such that each job requests a s

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Kulkarni Gururaj
• Vijayakumar Bala

Classifications

• G06F2009/45583

  Memory management, e.g. access or allocation · CPC title

• G06F2009/45595

  Network integration; Enabling network access in virtual machine instances · CPC title

• H04L69/40

  for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection (management of faults, events, alarms or notifications in data switching networks H04L41/06) · CPC title

• G06F11/1464 Primary

  for networked environments · CPC title

• G06F11/1453

  using de-duplication of the data · CPC title