Method and apparatus for reducing overheads of primary storage while transferring modified data

Patent metadata

Field	Value
Publication number	US-10176183-B1
Application number	US-201213665884-A
Country	US
Kind code	B1
Filing date	Oct 31, 2012
Priority date	Oct 31, 2012
Publication date	Jan 8, 2019
Grant date	Jan 8, 2019

Abstract

Techniques for reducing overheads of primary storage while transferring modified data are described. According to one embodiment, in response to a request at a primary storage for a backup of a plurality of data blocks, a snapshot of the plurality of data blocks is captured, wherein the snapshot represents a consistent state of the plurality of data blocks. Then a set of data block of the captured snapshot that are to be copied to a secondary storage is identified and the identified set of data blocks are transferred to a secondary storage over a network. In response to receiving data to be written to a first data block at the first primary storage during the transfer, current contents of the first data block is preserved if the first data block is in identified set of data blocks, and the received data is committed to the first data block.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving by a primary storage, at a periodic backup interval, a request to back up a plurality of data blocks from the primary storage to a secondary storage;
in response to receiving the request:
capturing a snapshot of the plurality of data blocks from a first location on the primary storage and transferring dirty blocks within the snapshot of the plurality of data blocks to a second location, wherein the plurality of data blocks in the first location comprise both normal and dirty blocks, wherein the snapshot comprises a copy of the plurality of data blocks that represents a consistent state of the plurality of data blocks on the primary storage at the first location at a point in time;
for each data block in the snapshot of the plurality of data blocks, marking the data block normal in response to determining that the data block has not changed since a last backup of the data block, otherwise marking the data block as dirty;
identifying a set of data blocks of the captured snapshot that are to be copied to the secondary storage, wherein the set of data blocks comprise only dirty data blocks;
determining a transfer order of the identified set of dirty data blocks that is different from a sequential logical order of the identified set of dirty data blocks provided by the primary storage, wherein a dirty data block of the identified set of dirty data blocks is given a higher priority for transfer based on proximity of the dirty data block to a frequently changed region on the primary storage;
transferring, in a transfer period shorter than the periodic backup interval, the identified set of dirty data blocks to the secondary storage over a network in the determined transfer order, wherein transferring the identified set of dirty data blocks to the secondary storage comprises, for each data block in the identified set of dirty data blocks of the snapshot, transferred during the transfer period:
transferring the data block to the secondary storage, and
marking the data block as normal in the first location of the primary storage of the plurality of data blocks in response to transferring the data block to the secondary storage;

in response to receiving data to be written to a first data block at the primary storage at the first location and determining that the first data block is dirty, preserving current contents of the first data block and committing the received data to the first data block at the first location of the first data block on the primary storage;
in response to receiving data to be written to the first data block at the primary storage at the first location and determining that the first data block is normal, committing the received data to the first data block at the first location of the primary storage without preserving the contents of the first data block.

2. The method of claim 1 , further comprising:
removing a data block from the identified set of data blocks after the data block has been copied to the secondary storage.

3. The method of claim 1 , wherein the identified set of data blocks to be copied includes all the data blocks of the captured snapshot.

4. The method of claim 1 , wherein the identified set of data blocks to be copied are data blocks that have been changed since a previous backup.

5. The method of claim 1 , wherein preserving the current contents of the first data block includes copying the current contents of the first data block to an alternate data block in the primary storage.

6. The method of claim 1 , wherein preserving the current contents of the first data block includes writing the received data to an alternate data block in the primary storage.

7. The method of claim 1 , wherein markings of the data blocks are stored in a bit vector having a plurality of bits, each corresponding to one of the data blocks, wherein a bit having a first logical value indicates that the corresponding data block is dirty, and wherein a bit having a second logical value indicates that the corresponding data block is normal.

8. The method of claim 7 , wherein marking of dirty data blocks of the snapshot from the transfer state to the normal state is performed after all data blocks in the identified set have been transferred from the primary storage to the secondary storage.

9. The method of claim 1 , wherein the first location and second location are both on the primary storage.

10. The method of claim 1 , further comprising:
during a second transfer period of a second backup interval:
capturing a second snapshot of a second plurality of data blocks;
for each data block in the second snapshot of the second plurality of data blocks, marking the data block normal in response to determining that the data block has not changed since a last backup of the data block, otherwise making the data block as dirty;
identifying a second set of data blocks of the captured second snapshot that are to be copied to the secondary storage, wherein the second set of data blocks comprise only dirty data blocks;
in response to determining that the first data block is in the captured second snapshot of dirty blocks to be copied to the secondary storage, during the second transfer period, transferring the first data block before transferring the other data blocks in the captured second snapshot.

11. The method of claim 1 , wherein the request to back up the plurality of blocks comprises a file on the primary storage.

12. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform a method, the method comprising:
receiving by a primary storage, at a periodic backup interval, a request to back up a plurality of data blocks from the primary storage to a secondary storage;
in response to receiving the request:
capturing a snapshot of the plurality of data blocks from a first location on the primary storage and transferring dirty blocks within the snapshot of the plurality of data blocks to a second location, wherein the plurality of data blocks in the first location comprise both normal and dirty data blocks, wherein the snapshot comprises a copy of the plurality of data blocks that represents a consistent state of the plurality of data blocks on the primary storage at the first location at a point in time;
for each data block in the snapshot of the plurality of data blocks, marking the data block normal in response to determining that the data block has not changed since a last backup of the data block, otherwise marking the data block as dirty;
identifying a set of data blocks of the captured snapshot that are to be copied to the secondary storage, wherein the set of data blocks comprise only dirty data blocks;
determining a transfer order of the identified set of dirty data blocks that is different from a sequential logical order of the identified set of dirty data blocks provided by the primary storage, wherein a dirty data block of the identified set of dirty data blocks is given a higher priority for transfer based on proximity of the dirty data block to a frequently changed region on the primary storage;
transferring, in a transfer period shorter than the periodic backup interval, the identified set of data blocks to the secondary storage over a network in the determined transfer order, wherein transferring the identified set of dirty data blocks to the secondary storage comprises,
for each data block in the identified set of dirty data blocks of the snapshot being transferred during the transfer period:
transferring the data block to the secondary storage, and
marking the data block as nor

Assignees

• Emc Ip Holding Co Llc

Inventors

• Shim Hyong
• Shilane Philip N
• Hsu Windsor W

Classifications

• G06F3/065

  Replication mechanisms · CPC title

• G06F11/1453

  using de-duplication of the data · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1458

  Management of the backup or restore process · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title