Distributed file system snapshot

Patent metadata

Field	Value
Publication number	US-2016246803-A1
Application number	US-201615097034-A
Country	US
Kind code	A1
Filing date	Apr 12, 2016
Priority date	Mar 3, 2014
Publication date	Aug 25, 2016
Grant date	—

Abstract

Technology is disclosed for managing data in a distributed file system (“the technology”). The technology can gather metadata information associated with the data stored within the distributed file system, create a secondary namespace within a local file system of a local host using the gathered metadata information and store the gathered metadata information as files within the secondary namespace. Further, when a request to create a PPI of the distributed file system is received, the technology can create a PPI of the secondary namespace using a PPI creation feature of the local file system.

First claim

1 . A computer implemented method, comprising:
receiving metadata information associated with data stored within a distributed file system having a namespace, the namespace maintaining a hierarchy of files and directories associated with the stored data; creating a secondary namespace within a local file system using the metadata information, a hierarchy of files and directories in the secondary namespace corresponding to the hierarchy of files and directories in the namespace of the distributed file system; and creating an image of the secondary namespace when a request to create an image of the distributed file system is received, the image of the secondary namespace created using an image creation feature of the local file system.

2 . The method of claim 1 , further comprising:
blocking a request to delete any portion of the stored data associated with a file included in the image of the secondary namespace.

3 . The method of claim 1 , further comprising:
receiving a request to store an additional data within the distributed file system; storing an additional metadata information generated from processing the received request in a data log, the additional metadata information including any changes to the hierarchy of files and directories in the namespace of the distributed file system; and modifying the secondary namespace using the additional metadata information stored within the data log, the modification including any change to the hierarchy of files and directories in the secondary namespace, the modification including storing a portion of the additional metadata information using the secondary namespace.

4 . The method of claim 1 , further comprising:
receiving a request to revert the distributed file system and the data stored within to an earlier state; identifying a particular image of the secondary namespace associated with the earlier state of the distributed file system; and generating an earlier state namespace of the distributed file system using the metadata information captured within the particular image.

5 . The method of claim 1 , wherein the distributed file system is a Hadoop Distributed File System (“HDFS”), wherein the local file system is a Network File System with a PPI creation feature.

6 . The method of claim 1 , wherein the local file system is a B-tree File System (“Btrfs”), the Btrfs including a PPI creation feature.

7 . A Distributed File System (“DFS”), comprising:
a component configured to receive a block map file and a namespace file associated with a namespace of the DFS, the block map file including a mapping between a data block associated a selected file stored using the DFS and a location of the data block associated with the selected file, the namespace file including information relating to a hierarchy of files and directories in the namespace of the DFS;
a component configured to determine the hierarchy of files and directories in the namespace of the DFS using the namespace file;
a component configured to gather a metadata information associated with the files and the directories using the namespace file and the block map file;
a component configured to create a secondary namespace within a local file system of a local host, a hierarchy of files and directories in the secondary namespace corresponding to the hierarchy of files and directories in the namespace of the DFS; and
a component configured to create a an image of the secondary namespace when a request to create an image of the DFS is received, the image of the secondary namespace created using an image creation feature of the local file system.

8 . The DFS of claim 7 , further comprising:
a component configured to block a request to delete any portion of the stored data associated with a file included in the image of the secondary namespace.

9 . The DFS of claim 7 , further comprising:
a component configured to receive a request to store an additional data within the distributed file system; a component configured to store an additional metadata information generated from processing the received request in a data log, the additional metadata information including any changes to the hierarchy of files and directories in the namespace of the distributed file system; and a component configured to modify the secondary namespace using the additional metadata information stored within the data log, the modification including any change to the hierarchy of files and directories in the secondary namespace, the modification including storing a portion of the additional metadata information using the secondary namespace.

10 . The DFS of claim 7 , further comprising:
a component configured to receive a request to revert the distributed file system and the data stored within to an earlier state; a component configured to identify a particular image of the secondary namespace associated with the earlier state of the distributed file system; and a component configured to generate an earlier state namespace of the distributed file system using the metadata information captured within the particular image.

11 . The DFS of claim 7 , wherein the local file system is a Network File System with an image creation feature.

12 . The DFS of claim 7 , wherein the local file system is a B-tree File System (“Btrfs”), the Btrfs including an image creation feature.

13 . A non-transitory computer readable storage medium storing computer executable instructions, comprising:
instructions for receiving metadata information associated with a data stored within a distributed file system, the namespace maintaining a hierarchy of files and directories associated with the stored data; instructions for creating a secondary namespace within a local file system using the metadata information, a hierarchy of files and directories in the secondary namespace corresponding to the hierarchy of files and directories in the namespace of the distributed file system; and instructions for creating an image of the secondary namespace when a request to create an image of the distributed file system is received, the image of the secondary namespace created using an image creation feature of the local file system.

14 . The computer readable storage medium of claim 13 , further comprising:
instructions for blocking a request to delete any portion of the stored data associated with a file included in the image of the secondary namespace.

15 . The computer readable storage medium of claim 13 , further comprising:
instructions for receiving a request to store an additional data within the distributed file system; instructions for storing an additional metadata information generated from processing the received request in a data log, the additional metadata information including any changes to the hierarchy of files and directories in the namespace of the distributed file system; and instructions for modifying the secondary namespace using the additional metadata information stored within the data log, the modification including any change to the hierarchy of files and directories in the secondary namespace, the modification including storing a portion of the additional metadata information using the secondary namespace.

16 . The computer readable storage medium of claim 13 , further comprising:
instructions for receiving a request to revert the distributed file system and the data stored within to an earlier state; instructions for identifying a particular image of the secondary namespace associated with the earlier state of the distributed file system; and instructions for generating an earli

Assignees

• Netapp Inc

Inventors

• Makkar Gaurav
• Narayanamurthy Srinivasan
• Muthyala Kartheek

Classifications

• G06F17/302

  Physics · mapped topic

• G06F17/30327

  Physics · mapped topic

• G06F17/30088 Primary

  Physics · mapped topic

• G06F17/30212

  Physics · mapped topic

• G06F16/2246

  Trees, e.g. B+trees · CPC title