Using interoperability between components of a data storage management system and appliances for data storage and deduplication to generate secondary and tertiary copies

Patent metadata

Field	Value
Publication number	US-2020167091-A1
Application number	US-201816201856-A
Country	US
Kind code	A1
Filing date	Nov 27, 2018
Priority date	Nov 27, 2018
Publication date	May 28, 2020
Grant date	—

Abstract

Illustrative storage manager and media agent are enhanced to interoperate with deduplication appliances. Advantages are realized when making secondary and tertiary copies and also when restoring from a deduplication appliance. Tiered indexing minimizes how much data is retained and stored at media agents. Tiered indexing enables media agents to efficiently extract needed information from deduplication appliances to make tertiary copies and to restore backed up copies. Interoperability techniques include media agents generating separate data streams to the deduplication appliance. Each data stream carries a different kind of data, e.g., payload data, metadata content, or high-level index information. On initial backup, the media agent instructs the deduplication appliance to deduplicate the payload data stream but not the other data streams, thus intelligently applying resources to data most likely to benefit from deduplication. For tertiary copies (copies of pre-existing copies at the deduplication appliance), the media agent avoids handling payload data altogether.

First claim

1 . A method for using a deduplication appliance to generate and store tertiary copies based on secondary copies, the method comprising:
by a media agent that executes on a first computing device comprising one or more processors and computer memory, receiving instructions from a storage manager to generate a synthetic-full copy from a plurality of secondary copies that are stored in the deduplication appliance in communication with the media agent, wherein the deduplication appliance comprises one or more data storage devices and is capable of self-managed deduplication; by the media agent, extracting from an associated index at the first computing device information about the plurality of secondary copies,
wherein the information comprises pointers to a plurality of second data chunks stored without deduplication at the deduplication appliance,
wherein a given second data chunk comprises metadata for a corresponding secondary copy among the plurality of secondary copies, and further comprises pointers to one or more first data chunks stored with deduplication at the deduplication appliance, and
wherein a given first data chunk comprises payload data of the corresponding secondary copy among the plurality of secondary copies;
by the media agent, determining which first data chunks and which metadata from the second data chunks to be included in the synthetic-full copy; by the media agent, generating a first data stream transmitted to the deduplication appliance, wherein the first data stream comprises a third data chunk comprising the metadata from the second data chunks determined by the media agent to be included in the synthetic-full copy, and further comprises pointers to one or more first data chunks determined by the media agent to be included in the synthetic-full copy; by the media agent, generating a second data stream transmitted to the deduplication appliance, wherein the second data stream comprises a fourth data chunk comprising a pointer to a corresponding third data chunk in the first data stream; by the media agent, instructing the deduplication appliance to generate and store the synthetic-full copy based on payload data in the one or more first data chunks and further based on the metadata in the third data chunk; and wherein the synthetic-full copy is generated and stored at the deduplication appliance without passing payload data through the media agent.

2 . The method of claim 1 , wherein at least some of the plurality of secondary copies are stored at the deduplication appliance in deduplicated form.

3 . The method of claim 1 , wherein the secondary copies and the synthetic-full copy are stored at the deduplication appliance in deduplicated form.

4 . The method of claim 1 , wherein the media agent instructs the deduplication appliance to apply deduplication to the payload data in the one or more first data chunks, and further instructs the deduplication appliance not to apply deduplication to the first data stream and to the second data stream.

5 . The method of claim 1 , wherein the synthetic-full copy is stored at the deduplication appliance in deduplicated form based on the media agent instructing the deduplication appliance to apply deduplication to the payload data in the one or more first data chunks, and further instructing the deduplication appliance not to apply deduplication to the first data stream and to the second data stream.

6 . The method of claim 1 , wherein the synthetic-full copy is stored at the deduplication appliance in deduplicated form, and further wherein no deduplication is applied by the media agent to the first data stream and the second data stream.

7 . The method of claim 1 further comprising:
by the media agent, instructing the deduplication appliance to store the synthetic-full copy to the one or more data storage devices in the deduplication appliance.

8 . The method of claim 1 further comprising:
by the media agent, instructing the deduplication appliance to store the synthetic-full copy to the one or more data storage devices in the deduplication appliance; and
by the media agent, updating the associated index to track the synthetic-full copy stored at the deduplication appliance.

9 . A method for using a deduplication appliance to generate and store tertiary copies based on secondary copies, the method comprising:
by a media agent that executes on a first computing device comprising one or more processors and computer memory, receiving instructions from a storage manager to generate an auxiliary copy from a secondary copy that is stored in the deduplication appliance in communication with the media agent, wherein the deduplication appliance comprises one or more data storage devices and is capable of self-managed deduplication; by the media agent, extracting from an associated index at the first computing device a storage location of the secondary copy in the deduplication appliance,
wherein a second data chunk at the storage location comprises metadata for the secondary copy, and further comprises pointers to one or more first data chunks stored with deduplication at the deduplication appliance, and
wherein a given first data chunk comprises payload data of the secondary copy;
by the media agent, determining that the one or more first data chunks and the metadata from the second data chunk are to be copied to the auxiliary copy; by the media agent, generating a first data stream transmitted to the deduplication appliance, wherein the first data stream comprises a third data chunk comprising the metadata from the second data chunk, and further comprises pointers to the one or more first data chunks; by the media agent, generating a second data stream transmitted to the deduplication appliance, wherein the second data stream comprises a fourth data chunk comprising a pointer to the third data chunk in the first data stream; by the media agent, instructing the deduplication appliance to generate and store the auxiliary copy based on payload data in the one or more first data chunks and further based on the metadata in the third data chunk; and wherein the auxiliary copy is generated and stored at the deduplication appliance without passing payload data through the media agent.

10 . The method of claim 9 , wherein the secondary copy and the auxiliary copy are stored at the deduplication appliance in deduplicated form.

11 . The method of claim 9 , wherein the media agent instructs the deduplication appliance to apply deduplication to the payload data in the one or more first data chunks, and further instructs the deduplication appliance not to apply deduplication to the first data stream and to the second data stream.

12 . The method of claim 9 , wherein the auxiliary copy is stored at the deduplication appliance in deduplicated form based on the media agent instructing the deduplication appliance to apply deduplication to the payload data in the one or more first data chunks, and further instructing the deduplication appliance not to apply deduplication to the first data stream and to the second data stream.

13 . The method of claim 9 , wherein the auxiliary copy is stored at the deduplication appliance in deduplicated form, and further wherein no deduplication is applied by the media agent to the first data stream and the second data stream.

14 . The method of claim 9 , further comprising:
by the media agent, instructing the deduplication appliance to store the auxiliary copy to the one or more data storage devices in the deduplication appliance.

15 . The method of claim 9 , further comprising:
by the media agent, instruct

Assignees

• Commvault Systems Inc

Inventors

• Haridas Ganesh
• Vijayan Manoj Kumar

Classifications

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title

• G06F3/0641 Primary

  De-duplication techniques · CPC title

• G06F3/0614

  Improving the reliability of storage systems · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title