Intelligent local management of data stream throttling in secondary-copy operations

Patent metadata

Field	Value
Publication number	US-2016248676-A1
Application number	US-201514630139-A
Country	US
Kind code	A1
Filing date	Feb 24, 2015
Priority date	Feb 24, 2015
Publication date	Aug 25, 2016
Grant date	—

Abstract

Intelligent local management of data stream throttling in data movement operations, such as secondary-copy operations in a storage management system, is disclosed. A local throttling manager may intelligently interoperate with co-resident data agents and/or a media agent executing on any given local computing device, whether a client computing device or a secondary storage computing device. The local throttling manager may allocate and manage the available bandwidth for various jobs and their constituent data streams—across the data agents and/or media agent. Bandwidth is dynamically allocated and re-allocated to data streams used by ongoing jobs, in response to new jobs starting and old jobs completing, without having to pause and restart ongoing jobs to accommodate bandwidth adjustments. The illustrative embodiment also provides local users with a measure of control over data streams—to suspend, pause, and/or resume them—independently from the centralized storage manager that manages the storage management system as a whole.

First claim

What is claimed is:

1 . A method for managing data streams that originate from a client computing device in a storage management system, the method comprising:
managing, by a throttling manager that executes on the client computing device, a bandwidth available for data streams that originate from the client computing device, wherein the managing comprises:
receiving, by the throttling manager, one or more respective registrations for one or more first data streams to be used, by a first data agent that also executes on the client computing device, for a first secondary-copy job relative to first primary data accessible to the client computing device,
wherein the first secondary-copy job transfers the first primary data to a first media agent that executes on a secondary storage computing device that is distinct from the client computing device;

receiving, by the throttling manager, one or more respective registrations for one or more second data streams to be used, by a second data agent that also executes on the client computing device, for a second secondary-copy job relative to second primary data accessible to the client computing device,
wherein the second secondary-copy job transfers the second primary data to the first media agent;

allocating, by the throttling manager, based on the bandwidth available for the received registrations, (i) first respective bandwidth values for the one or more first data streams to be used by the first secondary-copy job and (ii) second respective bandwidth values for the one or more second data streams to be used for the second secondary-copy job;
after the second secondary-copy job completes, (a) receiving, by the throttling manager, one or more respective de-registrations for the one or more second data streams and (b) re-allocating, by the throttling manager, based on the de-registrations and the maximum bandwidth available for the remaining registrations, third respective bandwidth values for the one or more first data streams,
wherein the first secondary-copy job continues to execute using the one or more first data streams at the re-allocated third respective bandwidth values which differ from the first respective bandwidth values.

2 . The method of claim 1 wherein the managing further comprises:
receiving, by the throttling manager, one or more respective registrations for one or more third data streams to be used, by a third data agent that also executes on the client computing device, for a third secondary-copy job relative to third primary data accessible to the client computing device, wherein the third secondary-copy job transfers the third primary data to the first media agent; and
based on the bandwidth available for the received registrations, (i) allocating, by the throttling manager, fourth respective bandwidth values for the one or more third data streams, and (ii) re-allocating, by the throttling manager, fifth respective bandwidth values for the one or more first data streams, wherein the first secondary-copy job continues to execute using the one or more first data streams at the re-allocated fifth respective bandwidth values which differ from the third respective bandwidth values.

3 . The method of claim 1 wherein the managing comprises maximizing, by the throttling manager, based on the maximum bandwidth, the respective bandwidth values allocated and re-allocated to registered data streams.

4 . The method of claim 1 wherein the throttling manager allocates and re-allocates bandwidth values to respective data streams registered by respective data agents that execute on the client computing device so that the total bandwidth used by the registered data streams is maximized relative to the maximum bandwidth available from the client computing device.

5 . The method of claim 1 further comprising:
receiving, by the throttling manager from the first data agent, in the course of the first secondary-copy job, a value for the amount of data transferred so far by the secondary-copy job.

6 . A method for user control of data transfers that originate from a client computing device in a storage management system, the method comprising:
receiving, a first input from a user associated with the client computing device,
wherein the first input requests a change relative to a bandwidth allocation for a first data transfer for the first client computing device,
wherein the first data transfer relates to a first copy operation to transfer at least part of first data, accessible to the client computing device, to a secondary storage computing device, and
wherein the secondary storage device is distinct from the client computing device;
managing on the client computing device a selected bandwidth available for multiple data transfers that originate from the client computing device,
wherein the multiple data transfers comprises the first data transfer, and
wherein the managing is based on a respective registration or respective de-registration received for at least one data transfer in the multiple data transfers; and
in response to the first input, causing the change request relative to the first data transfer, by the client computing device.

7 . The method of claim 6 wherein the change request comprises at least one of:
(i) suspending the first data transfer based on a supplied de-registration message,
(ii) pausing the first data transfer based on a supplied de-registration message, and
(iii) resuming the first data transfer based on a supplied registration message.

8 . The method of claim 6 wherein the change request comprises suspending the data transfer, which comprises (i) transmitting to a throttling manager a de-registration of the first data transfer and further comprises (ii) causing the first data transfer to permanently stop operating in the first copy operation.

9 . The method of claim 6 wherein the change request comprises pausing the first data transfer, which comprises (i) transmitting to a throttling manager a de-registration of the first data transfer and further comprises (ii) causing the first data transfer to stop operating in the first copy operation until a second input from the user indicates that the first data transfer should resume.

10 . The method of claim 6 wherein the change request comprises resuming the first data transfer, which comprises (i) transmitting to a throttling manager a registration for the first data transfer and further comprises (ii) causing the first data transfer to begin operating in the first copy operation.

11 . The method of claim 6 wherein the first action relative to the first data transfer is based on the user input received by the client computing device and is not based on instructions from a storage manager that manages the first copy operation in the storage management system.

12 . At least one tangible computer-readable medium carrying instructions, which when executed by a least one data processor, performs a method for management of data stream throttling in a storage management system, the method comprising:
receiving, by a first data agent that executes on a first client computing device, instructions for a first copy job relative to first primary data associated with the first data agent,
wherein the instructions comprise one or more parameters for one or more first data streams that are to be used for transferring the first primary data to a first media agent during the first copy job;
transmitting, by the first data agent, to a throttling manager, one or more respective registrations for the one or more first data streams,
wherein the throttling manager also executes on the first client computing device;
re

Assignees

• Commvault Systems Inc

Inventors

• Thanasekaran Vijayan

Classifications

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0646

  Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0647

  Migration mechanisms · CPC title