Automated pinning of file system subtrees

Patent metadata

Field	Value
Publication number	US-11645266-B2
Application number	US-202016992694-A
Country	US
Kind code	B2
Filing date	Aug 13, 2020
Priority date	Aug 13, 2020
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

Methods and systems for improved pinning of file system subtrees are provided. In one aspect, a method is provided that includes receiving an identifier of a base directory within a file system tree. A plurality of subnodes of the base directory may be identified within the file system tree. At least a subset of the subnodes may be temporarily pinned to a plurality of metadata servers (MDSs). Pinning each respective subnode of the at least a subset of subnodes may include hashing an identifier of the respective subnode to generate a hashed value corresponding to a particular MDS and assigning the particular MDS to store and manage metadata for a subdirectory associated with the respective subnode.

First claim

The invention claimed is:

1. A method comprising:
receiving an identifier of a base directory within a file system tree, wherein the base directory is associated with a random policy;
identifying a plurality of subnodes of the base directory within the file system tree; and
temporarily pinning at least a subset of the subnodes to a plurality of metadata servers (MDSs) by, for each respective subnode of the at least a subset of the subnodes:
hashing an identifier of the respective subnode to generate a hashed value corresponding to a particular MDS; and
assigning the particular MDS to store and manage metadata for a subdirectory associated with the respective subnode,
wherein the subdirectory is a child directory of the base directory, wherein the random policy is configured to randomly pin a plurality of child directories of the base directory.

2. The method of claim 1 , wherein the particular MDS is temporarily assigned to manage metadata for the subdirectory.

3. The method of claim 2 , wherein the particular MDS ceases to be assigned to store and manage metadata for the subdirectory based on one or more of a time constraint and a resource constraint.

4. The method of claim 3 , wherein the resource constraint includes (i) removing the assignment of the particular MDS when an amount of available memory for the MDS falls below a predetermined threshold and (ii) removing the assignment of the particular MDS when a cache of the particular MDS is at least partially cleared.

5. The method of claim 3 , wherein the time constraint includes at least one of (i) a predetermined amount of time passing after assigning the particular MDS to store and manage metadata for the subdirectory and (ii) a predetermined amount of time passing after receiving a request for metadata associated with the subdirectory.

6. The method of claim 1 , wherein the particular MDS ceases to be assigned to store and manage metadata for the subdirectory after receiving a request that identifies (i) the respective subnode or a parent node of the respective subnode and (ii) a different MDS from among the plurality of MDSs.

7. The method of claim 1 , wherein the hashing is a consistent hashing operation performed based on a quantity of the plurality of MDSs.

8. The method of claim 7 , wherein the identifier of the respective subnode is a numerical identifier of the respective subnode within the file system tree.

9. The method of claim 8 , wherein the numerical identifier is an inode identifier of the respective subnode within the file system tree.

10. The method of claim 1 , wherein assigning the particular MDS to store and manage metadata for the subdirectory includes updating directory metadata associated with the subdirectory.

11. The method of claim 1 , further comprising receiving a request including an identifier of the base directory and identifying at least one of (i) a distributed policy and (ii) a conditional distributed policy.

12. The method of claim 1 , wherein the plurality of MDSs are implemented as part of a metadata cluster assigned to manage metadata for the file system tree.

13. A system comprising:
a processor; and
a memory storing instructions which, when executed by the processor, cause the processor to:
receive an identifier of a new directory within a file system tree;
perform a probabilistic check to determine whether a subnode corresponding to the new directory should be pinned, wherein the new directory is a child directory of a base directory associated with a random policy, wherein the random policy is configured to randomly pin a proportion of a plurality of child directories of the base directory; and
responsive to determining that the subnode should be pinned, temporarily pin the subnode to a particular metadata server (MDS) from among a plurality of MDSs by:
hashing an identifier of the subnode to generate a hashed value corresponding to the particular MDS; and
assigning the particular MDS to store and manage metadata for the new directory.

14. The system of claim 13 , wherein the new directory is at least one of a newly-created directory within the file system tree and a newly-loaded directory within the file system tree.

15. The system of claim 13 , wherein the probabilistic check includes randomly generating a number and comparing the number to the proportion.

16. The system of claim 13 , wherein the particular MDS ceases to be assigned to store and manage metadata for the new directory based on one or more of a time constraint and a resource constraint.

17. The system of claim 13 , wherein the particular MDS ceases to be assigned to store and manage metadata for the new directory after receiving a request that identifies (i) the subnode corresponding to the new directory or a parent node of the subnode and (ii) a different MDS from among the plurality of MDSs.

18. The system of claim 13 , wherein the hashing is a consistent hashing operation performed based on a quantity of the plurality of MDSs and an inode identifier of the subnode within the file system tree.

19. A non-transitory, computer-readable medium storing instructions which, when executed by a processor, cause the processor to:
receive an identifier of a new directory within a file system tree;
perform a probabilistic check to determine whether a subnode corresponding to the new directory should be pinned, wherein the new directory is a child directory of a base directory associated with a random policy, wherein the random policy is configured to randomly pin a proportion of a plurality of child directories of the base directory; and
responsive to determining that the subnode should be pinned, temporarily pin the subnode to a particular metadata server (MDS) from among a plurality of MDSs by:
hashing an identifier of the subnode to generate a hashed value corresponding to the particular MDS; and

assigning the particular MDS to store and manage metadata for the new directory.

Assignees

• Red Hat Inc

Inventors

• Donnelly Patrick
• Anupkrishnan Sidharth

Classifications

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/182 Primary

  Distributed file systems · CPC title