Dynamic parallel save streams

Patent metadata

Field	Value
Publication number	US-9690809-B1
Application number	US-201314091237-A
Country	US
Kind code	B1
Filing date	Nov 26, 2013
Priority date	Nov 26, 2013
Publication date	Jun 27, 2017
Grant date	Jun 27, 2017

Abstract

Exemplary methods for dynamically performing multiple save streams include in response to receiving a request to perform backup of an aggregate backup data, splitting the backup data into savepoints based on a splitting policy. In one embodiment, the method includes creating a first set of directors, each director responsible for backing up one of the savepoints. The method further includes allocating save streams to each of the directors, wherein each save stream is used by the corresponding director for backing up its savepoint. In one embodiment, in response to receiving an indication from a first director of the first set of directors that its savepoint has been backed up, creating a second set of directors, and allocating one or more save streams from a number of save streams freed by the first director to a second (unfinished) director from the first set of directors.

First claim

What is claimed is:

1. A computer-implemented method for dynamically allocating save streams in a storage system having multiple save streams and a master controller, the method comprising:
in response to receiving a request to perform backup of an aggregate backup data, splitting the aggregate backup data into a plurality of savepoints based on a predetermined splitting policy;
creating a first set of one or more directors, wherein each director of the first set of one or more directors is responsible for backing up one of the plurality of savepoints;
allocating one or more save streams to each director of the first set of one or more directors, wherein each save stream represents one or more resources used by an associated director for backing up its savepoint;
determining that all of the multiple save streams have been allocated and are in use by the one or more directors, and determining that one or more savepoints of the plurality of save points have not yet been assigned to a director; and
in response to receiving an indication from a first director of the first set of one or more directors that its savepoint has been backed up and in response to receiving an indication that the one or more save streams allocated to the first director have been freed:
creating, by the master controller, a new director;
freeing the save streams allocated to the first director;
allocating the freed save streams of the first director to the new director;
backing up a savepoint of the one or more savepoints remaining to be backed up using the new director.

2. The method of claim 1 , wherein in response to determining that all of the save streams in the plurality of save streams have been allocated to the plurality of directors:
receiving an indication from a second director of the first set of one or more directors that its savepoint has been backed up, and in response to receiving an indication that the one or more save streams allocated to the second director have been freed:
allocating one or more save streams that have been freed by the second director to one or more of the directors of the first set of one or more directors.

3. The method of claim 2 , wherein a number of save streams allocated to each of the first set of one or more directors is based on a total number of available save streams.

4. The method of claim 3 , wherein the number of save streams allocated to each of the first set of one or more directors is further based on a predetermined streaming policy indicating a number save streams to be allocated to each director.

5. The method of claim 4 , wherein the number of save streams allocated to each of the first set of one or more directors is further based on size and density of each of the savepoints that the first set of one or more directors is responsible for backing up.

6. The method of claim 5 , wherein creating the second set of one or more directors is based on the predetermined streaming policy and a number of save streams that has been freed by the first director which completed backing up its savepoint.

7. The method of claim 1 , wherein the savepoint of the at least one of the directors in the first set has a greater density than the saveset of the another director in the first set.

8. A non-transitory computer-readable medium having computer instructions stored therein, which when executed by a processor, cause the processor to perform operations for dynamically allocating save streams in a storage system having multiple save streams and a master controller, the operations comprising:
in response to receiving a request to perform backup of an aggregate backup data, splitting the aggregate backup data into a plurality of savepoints based on a predetermined splitting policy;
creating a first set of one or more directors, wherein each director of the first set of one or more directors is responsible for backing up one of the plurality of savepoints;
allocating one or more save streams to each director of the first set of one or more directors, wherein each save stream represents one or more resources used by an associated director for backing up its savepoint;
determining that all of the multiple save streams have been allocated to the first set of directors and are in use by the one or more directors, and determining that one or more savepoints of the plurality of save points have not yet been assigned to a director; and
in response to receiving an indication from a first director of the first set of one or more directors that its savepoint has been backed up and in response to receiving an indication that the one or more save streams allocated to the first director have been freed:
creating, by the master controller, a new director;
freeing the save streams allocated to the first director;
allocating the freed save streams of the first director to the new director;
backing up a savepoint of the one or more savepoints remaining to be backed up using the new director.

9. The non-transitory computer-readable medium of claim 8 , wherein in response to determining that all of the save streams in the plurality of save streams have been allocated to the plurality of directors:
receiving an indication from a second director of the first set of one or more directors that its savepoint has been backed up, and in response to receiving an indication that the one or more save streams allocated to the second director have been freed:
allocating one or more save streams that have been freed by the second director to one or more of the directors of the first set of one or more directors.

10. The non-transitory computer-readable medium of claim 9 , wherein a number of save streams allocated to each of the first set of one or more directors is based on a total number of available save streams.

11. The non-transitory computer-readable medium of claim 10 , wherein the number of save streams allocated to each of the first set of one or more directors is further based on a predetermined streaming policy indicating a number save streams to be allocated to each director.

12. The non-transitory computer-readable medium of claim 11 , wherein the number of save streams allocated to each of the first set of one or more directors is further based on size and density of each of the savepoints that the first set of one or more directors is responsible for backing up.

13. The non-transitory computer-readable medium of claim 12 , wherein creating the second set of one or more directors is based on the predetermined streaming policy and a number of save streams that has been freed by the first director which completed backing up its savepoint.

14. The non-transitory computer-readable medium of claim 8 , wherein the savepoint of the at least one of the directors in the first set has a greater density that the saveset of the another director in the first set.

15. A data processing system, comprising:
a network interface configured to receive backup requests;
a master controller, coupled to the network interface, wherein the master controller comprises at least one hardware processor coupled to a memory programmed with executable instructions that, when executed by the at least one hardware processor, configure the master controller to:
in response to receiving a request to perform backup of an aggregate backup data, split the aggregate backup data into a plurality of savepoints based on a predetermined splitting policy;
create a first set of one or more directors, wherein each director of the first set of one or more directors is responsible for backing up one of the plurality of savepoints;
allocate one or more

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Chai Mu
• Mandic Vladimir
• Armorer Peter

Classifications

• G06F17/30289 Primary

  Physics · mapped topic

• G06F17/30091

  Physics · mapped topic

• G06F16/10 Primary

  File systems; File servers · CPC title

• G06F16/13

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title

• G06F16/21 Primary

  Design, administration or maintenance of databases · CPC title