Data recovery operations, such as recovery from modified network data management protocol data

Patent metadata

Field	Value
Publication number	US-9244779-B2
Application number	US-201113241625-A
Country	US
Kind code	B2
Filing date	Sep 23, 2011
Priority date	Sep 30, 2010
Publication date	Jan 26, 2016
Grant date	Jan 26, 2016

Abstract

The systems and methods herein permit storage systems to correctly perform data recovery, such as direct access recovery, of Network Data Management Protocol (“NDMP”) backup data that was modified prior to being stored in secondary storage media, such as tape. For example, as described in greater detail herein, the systems and methods may permit NDMP backup data to be encrypted, compressed, deduplicated, and/or otherwise modified prior to storage. The systems and methods herein also permit a user to perform a precautionary snapshot of the current state of data (e.g., primary data) prior to reverting data to a previous state using point-in-time data.

First claim

We claim:

1. A method of handling a Network Data Management Protocol (“NDMP”) backup or copy operation, the method comprising:
receiving an original NDMP backup data stream from a client;
modifying the received original NDMP backup data stream by performing the following modification procedures upon the original NDMP backup data stream: deduplication, encryption, and compression,
wherein the modification procedures create a modified version of the NDMP backup data stream;

storing the modified version of the NDMP backup data stream in stored chunks,
wherein each chunk is addressable by a physical offset and a physical length,
wherein each chunk represents a modified version of a contiguous subset of the original NDMP backup data stream, and,
wherein the client associates a logical offset and a logical length with the contiguous subset of the original NDMP backup data stream that is represented by each chunk; and,

for each stored chunk, associating the stored chunk with:
the physical offset and physical length that permits addressing of the chunk,
the logical offset and logical length that the client associates with the contiguous subset of the original NDMP backup data stream that is represented by the chunk, and

receiving a request for a certain subset of the original NDMP backup data stream, wherein the request includes a logical offset and a logical length;
using the received logical offset and logical length to:
identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and
determine a physical offset and a physical length that permits addressing of the identified chunk;

reading at least a portion of the identified chunk using the determined physical offset and physical length;
converting the portion of the identified chunk by reversing the modification procedures that were previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and,
sending data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.

2. The method of claim 1 , further comprising:
reading another portion of the identified chunk;
converting the other portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the other portion of the identified chunk;
buffering data from the converted version of the other portion of the identified chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

3. The method of claim 1 , further comprising:
reading at least a portion of another chunk that represents a modified version of a contiguous subset of the original NDMP backup data stream that is adjacent to at least a portion of the requested subset of the original NDMP backup data stream;
converting the portion of the other chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the portion of the other chunk;
buffering data from the converted version of the portion of the other chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and,
responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

4. The method of claim 1 , wherein modifying the received original NDMP backup data stream further comprises adding metadata to the data stream.

5. The method of claim 1 , wherein a storage policy specifies a configuration of the backup or copy operation.

6. The method of claim 1 , wherein associating a stored chunk with a physical offset, physical length, logical offset, and logical length comprises associating a chunk identifier with the physical offset, physical length, logical offset, and logical length in a chunk mapping table.

7. The method of claim 1 ,
wherein modifying the received original NDMP backup data stream comprises performing deduplication; and
wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.

8. The method of claim 1 ,
wherein modifying the received original NDMP backup data stream comprises performing deduplication; and
wherein at least one chunk corresponds to a logical subdivision of a deduplication data structure that is created by the performance of deduplication.

9. The method of claim 1 , wherein a storage policy specifies an approximate size of the stored chunks.

10. The method of claim 1 , wherein the size of stored chunks is configurable to tune performance of restore operations.

11. The method of claim 1 , further comprising:
determining that one or more stored chunks are being moved to a new physical location during a storage operation; and,
for each of the one or more moved chunks, associating the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.

12. A system for handling a Network Data Management Protocol (“NDMP”) backup or copy operation, the system comprising:
a processor; and,
a secondary storage computing device, coupled to the processor, and configured to:
receive an original NDMP backup data stream from a client;
modify the received original NDMP backup data stream by performing at least two the following modification procedures upon the original NDMP backup data stream: deduplication, encryption, and compression,
wherein the at least two modification procedures create a modified version of the NDMP backup data stream;

store the modified version of the NDMP backup data stream in stored chunks,
wherein each chunk is addressable by a physical offset and a physical length,
wherein each chunk represents a modified version of a subset of the original NDMP backup data stream, and,
wherein the client associates a logical offset and a logical length with the subset of the original NDMP backup data stream that is represented by each chunk; and,

for each stored chunk, associate the stored chunk with:
the physical offset and physical length that permits addressing of the chunk, and,
the logical offset and logical length that the client associates with the subset of the original NDMP backup data stream that is represented by the chunk;

receive a request for a certain subset of the original NDMP backup data stream,
wherein the request includes a logical offset and a logical length;

use the received logical offset and logical length to:
identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and
determine a physical offset and a physical length that permits addressing of the identified chunk;

read at least a portion of the identified chunk using the determined physical offset and physical length;
convert the portion of the identified chunk by reversing each of the modification procedures that were previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and,
sending data from the converted version of the portion of the ident

Assignees

• Littlefield Duncan A
• Nallathambi Vimal K
• Chanchlani Girish
• Commvault Systems Inc

Inventors

• Littlefield Duncan A
• Nallathambi Vimal K
• Chanchlani Girish

Classifications

• G06F11/1469 Primary

  Backup restoration techniques · CPC title

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F11/1448

  Management of the data involved in backup or backup restore · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0641

  De-duplication techniques · CPC title