Snapshot metadata arrangement for efficient cloud integrated data management

Patent metadata

Field	Value
Publication number	US-12099467-B2
Application number	US-202318333627-A
Country	US
Kind code	B2
Filing date	Jun 13, 2023
Priority date	Oct 28, 2016
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A storage appliance arranges snapshot data and snapshot metadata into different structures, and arranges the snapshot metadata to facilitate efficient snapshot manipulation, which may be for snapshot management or snapshot restore. The storage appliance receives snapshots according to a forever incremental configuration and arranges snapshot metadata into different types of records. The storage appliance stores these records in key-value stores maintained for each defined data collection (e.g., volume). The storage appliance arranges the snapshot metadata into records for inode information, records for directory information, and records that map source descriptors of data blocks to snapshot file descriptors. The storage appliance uses a locally generated snapshot identifier as a key prefix for the records to conform to a sort constrain of the key-value store, which allows the efficiency of the key-value store to be leveraged. The snapshot metadata arrangement facilitates efficient snapshot restore, file restore, and snapshot reclamation.

First claim

What is claimed is:

1. A method implemented by one or more processors and comprising:
identifying a key-value store for a data collection in one or more file systems maintained by one or more data source devices based on a data collection identifier included in a file restore request, wherein the file restore request further comprises a source snapshot identifier for a source snapshot and a path name that includes a file name for a file;
determining that the source snapshot is a baseline snapshot based on a comparison of the local snapshot identifier with a data structure associated with the data collection and storing parent-child relationships for one or more snapshots for the data collection;
retrieving first data map records from the key-value store using a local snapshot identifier for the source snapshot and an inode number for the file as a key prefix for each of the first data map records, wherein the inode number is obtained based on a search of one or more directory records of one or more snapshots based on the path name;
sending the file via a data storage network to a restore target identified in the file restore request, wherein the file is reconstructed by assembling data blocks from a snapshot data file for the source snapshot according to offsets in the first data map records that map to locations in the snapshot data file.

2. The method of claim 1 , further comprising assembling the data blocks from the snapshot data file for the source snapshot further according to lengths associated with the offsets in the first data map records.

3. The method of claim 1 , wherein the local snapshot identifier is mapped to the source snapshot identifier in stored metadata for the data collection.

4. The method of claim 1 , wherein the snapshot data file is identified using the local snapshot identifier, comprises an aggregation of file data for files associated with the source snapshot, and is retrieved from one or more cloud storage devices.

5. A non-transitory machine-readable medium having stored thereon instructions comprising machine executable code that, when executed by at least one machine, causes the machine to:
identify a key-value store for a data collection in one or more file systems maintained by one or more data source devices based on a data collection identifier included in a file restore request, wherein the file restore request further comprises a source snapshot identifier for a source snapshot and a path name that includes a file name for a file;
determine that the source snapshot is a baseline snapshot based on a comparison of the local snapshot identifier with a data structure associated with the data collection and storing parent-child relationships for one or more snapshots for the data collection;
retrieve first data map records from the key-value store using a local snapshot identifier for the source snapshot and an inode number for the file as a key prefix for each of the first data map records, wherein the inode number is obtained based on a search of one or more directory records of one or more snapshots based on the path name; and
send the file via a data storage network to a restore target identified in the file restore request, wherein the file is reconstructed by assembling data blocks from a snapshot data file for the source snapshot according to offsets in the first data map records that map to locations in the snapshot data file.

6. The non-transitory machine readable medium of claim 5 , wherein the machine executable code when executed by the machine further causes the machine to assemble the data blocks from the snapshot data file for the source snapshot further according to lengths associated with the offsets in the first data map records.

7. The non-transitory machine readable medium of claim 5 , wherein the local snapshot identifier is mapped to the source snapshot identifier in stored metadata for the data collection.

8. The non-transitory machine readable medium of claim 5 , wherein the snapshot data file is identified using the local snapshot identifier, comprises an aggregation of file data for files associated with the source snapshot, and is retrieved from one or more cloud storage devices.

9. A computing device, comprising:
memory containing non-transitory medium comprising machine executable code having stored thereon instructions; and
one or more processors coupled to the memory, the one or more processors configured to execute the machine executable code to cause the one or more processors to:
identify a key-value store for a data collection in one or more file systems maintained by one or more data source devices based on a data collection identifier included in a file restore request, wherein the file restore request further comprises a source snapshot identifier for a source snapshot and a path name that includes a file name for a file;
determine that the source snapshot is a baseline snapshot based on a comparison of the local snapshot identifier with a data structure associated with the data collection and storing parent-child relationships for one or more snapshots for the data collection;
retrieve first data map records from the key-value store using a local snapshot identifier for the source snapshot and an inode number for the file as a key prefix for each of the first data map records, wherein the inode number is obtained based on a search of one or more directory records of one or more snapshots based on the path name; and
send the file via a data storage network to a restore target identified in the file restore request, wherein the file is reconstructed by assembling data blocks from a snapshot data file for the source snapshot according to offsets in the first data map records that map to locations in the snapshot data file.

10. The computing device of claim 9 , wherein the one or more processors are further configured to execute the machine executable code to assemble the data blocks from the snapshot data file for the source snapshot further according to lengths associated with the offsets in the first data map records.

11. The computing device of claim 9 , wherein the local snapshot identifier is mapped to the source snapshot identifier in stored metadata for the data collection.

12. The computing device of claim 9 , wherein the snapshot data file is identified using the local snapshot identifier, comprises an aggregation of file data for files associated with the source snapshot, and is retrieved from one or more cloud storage devices.

13. The computing device of claim 9 , wherein the one or more processors are further configured to execute the machine executable code to merge data ranges that are valid into a restore data map prior to assembling the data blocks based on the restore data map and sending the file to the restore target.

14. The computing device of claim 13 , wherein the one or more processors are further configured to execute the machine executable code to determine when the data ranges are valid based on a comparison of the data ranges to the offsets and a plurality of lengths of the first data map records.

Assignees

• Netapp Inc

Inventors

• Jain Sharad
• Tapaswi Hrishikesh Ashok
• Zheng Ling
• Bhat Onkar
• Kaushik Akhil

Classifications

• G06F2201/84

  Using snapshots, i.e. a logical point-in-time copy of the data · CPC title

• G06F2201/805

  Real-time · CPC title

• G06F11/1469

  Backup restoration techniques · CPC title

• G06F11/1448 Primary

  Management of the data involved in backup or backup restore · CPC title

• G06F11/1464

  for networked environments · CPC title