Concurrent access and transactions in a distributed file system

Patent metadata

Field	Value
Publication number	US-9870368-B2
Application number	US-201414525018-A
Country	US
Kind code	B2
Filing date	Oct 27, 2014
Priority date	Oct 27, 2014
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

Embodiments described herein provide techniques for maintaining consistency in a distributed system (e.g., a distributed secondary storage system). According to one embodiment of the present disclosure, a first set of file system objects included in performing the requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into an inode associated with each identified file system object. Each file system object corresponding to the inode is modified as specified by the update intent in that inode. After modifying the file system object corresponding to the inode, the update intent is removed from that inode.

First claim

What is claimed is:

1. A method for managing consistency in a distributed file system, the method comprising:
determining an inode is associated with an incomplete file system operation based on an update intent stored at the inode, wherein the inode is associated with a file system object in the distributed file system, wherein the update intent stored at the inode indicates that the inode is part of a file system operation that includes a plurality of inodes, wherein the update intent stored at the inode specifies an ordered sequence of a plurality of inodes associated with the incomplete file system operation and a position of the inode in the ordered sequence with respect to the plurality of inodes;
retrieving, as available, a set of inodes specified by the ordered sequence;
determining which inodes of the set of inodes store a corresponding update intent related to the incomplete file system operation; and
selectively aborting the incomplete file system operation based on the determined inodes of the set of inodes that store the corresponding update intent related to the incomplete file system operation,
wherein the incomplete file operation is aborted in the event one or more inodes of the set of inodes that are before the inode in the ordered sequence include the corresponding update intent related to the incomplete file system operation and one or more inodes of the set of inodes that are after the inode in the ordered sequence do not include the corresponding update intent related to the incomplete file system operation.

2. The method of claim 1 , wherein:
aborting the incomplete file system operation includes removing the corresponding update intent related to the incomplete file system operation from each inode specified by the ordered sequence that includes the corresponding update intent.

3. The method of claim 1 , further comprising selectively completing the incomplete file system operation based on the determined nodes that store the corresponding update intent related to the incomplete file system operation, wherein the incomplete file system is completed in the event one or more inodes of the set of inodes that are before the inode in the ordered sequence do not include the corresponding update intent related to the incomplete file system operation and one or more inodes of the set of inodes that are after the inode in the ordered sequence include the corresponding update intent related to the incomplete file system operation.

4. The method of claim 3 , further comprising, removing the corresponding update intent from each of the one or more inodes of that set of inodes that are after the inode in the ordered sequence.

5. The method of claim 1 , wherein the distributed file system is exposed to clients as an NFS mount point, and wherein the incomplete file system operation is an NFS operation from a client sent to a node of the distributed file system.

6. The method of claim 1 , further comprising, prior to aborting the incomplete file system operation, obtaining a node-specific lock and a process-specific lock on each of the set of inodes.

7. The method of claim 6 , wherein the node-specific lock prevents other nodes in the distributed file system from accessing the locked inodes during the file system operation, and wherein the process-specific lock prevents other local processes from accessing the inodes.

8. The method of claim 3 , wherein completing the incomplete file system operation includes completing the incomplete file system operation as specified by the update intent and the corresponding update intent of the set of inodes that are after the inode in the ordered sequence.

9. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform an operation for managing consistency in a distributed file system, the operation comprising:
determining an inode is associated with an incomplete file system operation based on an update intent stored at the inode, wherein the inode is associated with a file system object in the distributed file system, wherein the update intent stored at the inode indicates that the inode is part of a file system operation that includes a plurality of inodes, wherein the update intent stored at the inode specifies an ordered sequence of a plurality of inodes associated with the incomplete file system operation and a position of the inode in the ordered sequence with respect to the plurality of inodes;
retrieving, as available, a set of inodes specified by the ordered sequence; and
determining which inodes of the set of inodes store a corresponding update intent related to the incomplete file system operation; and
selectively aborting the incomplete file system operation based the determined inodes of the set of inodes that store the corresponding update intent related to the incomplete file system operation,
wherein the incomplete file operation is aborted in the event one or more inodes of the set of inodes that are before the inode in the ordered sequence include the corresponding update intent related to the incomplete file system operation and one or more inodes of the set of inodes that are after the inode in the ordered sequence do not include the corresponding update intent related to the incomplete file system operation.

10. The computer-readable medium of claim 9 , wherein
aborting the incomplete file system operation includes removing the corresponding update intent related to the incomplete file system operation from each inode specified by the ordered sequence that includes the corresponding update intent.

11. The computer-readable medium of claim 9 , further comprising selectively completing the incomplete file system operation based on the determined nodes that store the corresponding update intent related to the incomplete file system operation, wherein the incomplete file system is completed in the event one or more inodes of the set of inodes that are before the inode in the ordered sequence do not include the corresponding update intent related to the incomplete file system operation and one or more inodes of the set of inodes that are after the inode in the ordered sequence include the corresponding update intent related to the incomplete file system operation.

12. The computer-readable medium of claim 11 , wherein the operation further comprises, removing the corresponding update intent from each of the one or more inodes of the set of inodes that are after the inode in the ordered sequence.

13. The computer-readable medium of claim 9 , wherein the distributed file system is exposed to clients as an NFS mount point, and wherein the incomplete file system operation is an NFS operation from a client sent to a node of the distributed file system.

14. The computer-readable medium of claim 9 , wherein the operation further comprises, prior to aborting the incomplete file system operation, obtaining a node-specific lock and a process-specific lock on each of the set of inodes.

15. The computer-readable medium of claim 11 , wherein completing the incomplete file system operation includes completing the incomplete file system operation as specified by the update intent and the corresponding update intent of one or more inodes of the set of inodes that are after the inode in the ordered sequence.

16. A system comprising:
a processor; and
a memory storing program code, which, when executed on the processor performs an operation for managing consistency in a distributed file system, the operation comprising:
determining an inode is associated with an incomplete file system

Assignees

• Cohesity Inc

Inventors

• Aron Mohit
• Shanmuganathan Ganesha

Classifications

• G06F17/30958

  Physics · mapped topic

• G06F17/30168 Primary

  Physics · mapped topic

• G06F16/9024

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06F16/1767 Primary

  Concurrency control, e.g. optimistic or pessimistic approaches · CPC title