Dynamic parallel save streams for block level backups

Patent metadata

Field	Value
Publication number	US-10069909-B1
Application number	US-201514974419-A
Country	US
Kind code	B1
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Sep 4, 2018
Grant date	Sep 4, 2018

Abstract

Embodiments are directed to performing block-based backups of data using dynamic save streams by parsing, in a backup manager component, a data saveset to convert the data into blocks, analyzing the blocks in a stream engine component to define a number and respective size of save streams to assign certain blocks to respective streams, and transmitting the save streams to a backup storage media to perform a backup operation.

First claim

What is claimed is:

1. A computer-implemented method for performing data backups in a data backup system having a backup manager computer and storage medium, comprising:
parsing, in the backup manager computer, a data saveset to convert the data into blocks to create block level backups using dynamic parallel save stream technology;
analyzing the blocks in a stream engine component of the backup manager computer to define a number and respective size of parallel save streams to assign certain blocks to respective streams in accordance with a parallelism policy;
determining, in a rule engine of the backup computer, whether to include at least some file level backups as part of the block level backups for the data saveset;
transmitting the save streams to a backup storage media, wherein the number of save streams is one of a default value or a user configurable value and is denoted N, and further wherein the stream engine component creates N worker threads, where each thread opens a file separately and moves the starting offset to a respective defined value, wherein each thread has no lock so that each thread gets a different file handle for the same file, and yet further wherein the respective defined value for a thread equals the file size multiplied by the result of the thread number divided by the thread count for the respective thread; and
storing the save streams in the storage medium to complete a backup operation.

2. The method of claim 1 wherein the blocks are one of a fixed length or a variable length.

3. The method of claim 2 further comprising:
creating logical blocks for the saveset, wherein a logical organization comprises the blocks starting from different offsets;
consolidating the blocks;
generating the number of save streams from the consolidated blocks; and
running the backup operation to save the saveset to the backup media storage.

4. The method of claim 3 wherein the backup operation comprises one of a full backup or an incremental backup, and the saveset represents a snapshot or consistent state image of a large database.

5. The method of claim 3 further comprising determining, in a rules engine, a type of backup to be performed based on certain defined rule criteria, the type of backup comprising one of a file level parallel save stream backup, a block level parallel save stream backup, and a combined file and block level parallel save stream backup.

6. The method of claim 5 wherein the defined rule criteria include a plurality of saveset characteristics comprised of at least: file size, file density, block size, and percentage change of data for incremental backups.

7. The method of claim 6 further comprising assigning respective weights to each characteristic of the plurality of saveset characteristics, and combining the rule criteria according to a defined formula to determine the backup type based on the plurality of weighted save set characteristics.

8. The method of claim 3 wherein contiguous blocks are assigned to a same respective save stream.

9. The method of claim 3 wherein the saveset is dynamically split into the save streams by one of: splitting into N logically contiguous sequential sections; or splitting into interleaved logically sized blocks.

10. A system configured to perform data backups across a network in a data backup system having a backup manager computer and storage medium, comprising:
a backup manager component parsing, in the backup manager computer, a data saveset to convert the data into blocks to create block level backups using dynamic parallel save stream technology, wherein the number of save streams is one of a default value or a user configurable value, and further wherein the number of save streams is denoted N, and yet further wherein the stream engine component creates N worker threads, where each thread opens a file separately and moves the starting offset to a respective defined value, wherein each thread has no lock so that each thread gets a different file handle for the same file, and further wherein the respective defined value for a thread equals the file size multiplied by the result of the thread number divided by the thread count for the respective thread;
a save stream engine coupled to the backup manager component and analyzing the blocks in a stream engine component to define a number and respective size of parallel save streams to assign certain blocks to respective streams in accordance with a parallelism policy;
an interface transmitting the save streams to a backup storage media; and
the backup storage media storing the transmitted save streams.

11. The system of claim 10 wherein the blocks are one of a fixed length or a variable length.

12. The system of claim 11 wherein the save stream engine further:
creates logical blocks for the saveset, wherein a logical organization comprises the blocks starting from different offsets;
consolidates the blocks and generates the number of save streams from the consolidated blocks; and
transmits the save streams in a backup operation to save the saveset to the backup media storage.

13. The system of claim 12 further comprising a rules engine determining a type of backup to be performed based on certain defined rule criteria, the type of backup comprising one of a file level parallel save stream backup, a block level parallel save stream backup, and a combined file and block level parallel save stream backup.

14. The system of claim 13 wherein the defined rule criteria include a plurality of saveset characteristics comprised of at least: file size, file density, block size, and percentage change of data for incremental backups.

15. The system of claim 14 further wherein the rules engine assigns respective weights to each characteristic of the plurality of saveset characteristics, and combines the rule criteria according to a defined formula to determine the backup type based on the plurality of weighted saveset characteristics.

16. The system of claim 15 wherein the backup operation comprises one of a full backup or an incremental backup, and the saveset represents a snapshot or consistent state image of a large database, and wherein the number of save streams is determined by a parallelism parameter defined in the backup manager.

17. A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to perform data backups in a data backup system having a backup manager computer and storage medium by executing instructions implementing a method comprising:
parsing, in the backup manager computer, a data saveset to convert the data into blocks to create block level backups using dynamic parallel save stream technology;
analyzing the blocks in a stream engine component of the backup manager computer to define a number and respective size of parallel save streams to assign certain blocks to respective streams in accordance with a parallelism policy;
determining, in a rule engine of the backup computer, whether to include at least some file level backups as part of the block level backups for the data saveset;
transmitting the save streams to a backup storage media, wherein the number of save streams is one of a default value or a user configurable value and is denoted N, and further wherein the stream engine component creates N worker threads, where each thread opens a file separately and moves the starting offset to a respective defined value, wherein each thread has no lock so that each

Assignees

• Emc Ip Holding Co Llc

Inventors

• Chopra Shelesh
• Mandic Vladimir
• Armorer Peter
• Kulkarni Gururaj

Classifications

• H04L65/80

  Responding to QoS · CPC title

• G06F11/1464

  for networked environments · CPC title

• H04L67/1095 Primary

  Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes · CPC title

• H04L65/4069

  Electricity · mapped topic