Partitioning a distributed filesystem namespace

Patent metadata

Field	Value
Publication number	US-2020250137-A1
Application number	US-201916265412-A
Country	US
Kind code	A1
Filing date	Feb 1, 2019
Priority date	Feb 1, 2019
Publication date	Aug 6, 2020
Grant date	—

Abstract

The described technology is generally directed towards domains that data services can use to collect files of a global filesystem namespace into groups. A data service (a domain patron) creates a domain, e.g., for a particular directory, and a domain manager associates files under that directory with domain membership information. Thereafter, the data service can use the domain membership information associated with a file to determine whether to include that file in a data service operation. In one implementation the membership information is maintained in file metadata, facilitating fast and efficient retrieval of the information in near constant time. Exclusion from a domain, hard links, renames and alternate data streams are among the various aspects supported by the domain technology.

First claim

What is claimed is:

1 . A method, comprising:
receiving, by a system comprising a processor, a request to create a domain on a directory of a filesystem; and in response to the request,
creating the domain on the directory,
associating the directory with the domain, and
in response to the directory not being empty, associating a node under the directory with the domain.

2 . The method of claim 1 , further comprising, in response to the request, storing an identifier of the domain in a domain data structure.

3 . The method of claim 1 , further comprising, in response to the request, updating a generation value in a domain data structure.

4 . The method of claim 1 , wherein the request to create the domain is a first request, wherein the node comprises a file object, and further comprising:
receiving a second request to obtain domain membership information with respect to the file object, in response to the second request, determining whether the domain membership information of the file object is up to date, in response to the domain membership information of the file object being determined to be up to date, returning the domain membership information, in response to the domain membership information of the file object being determined to be out of date, updating the domain membership information by obtaining updated domain membership information from a parent node of the file object, and returning the updated domain membership.

5 . The method of claim 1 , wherein the directory is not empty, and wherein the associating the node under the directory with the domain comprises tagging an attribute associated with the node with an identifier of the domain to designate that the node is a member of the domain.

6 . The method of claim 1 , wherein the directory is not empty, wherein the node comprises a file object, and wherein the associating the node under the directory with the domain comprises walking a directory hierarchy, and tagging a file attribute associated with the file object with an identifier of the domain to designate that the node is a member of the domain when the file object is reached during the walking.

7 . The method of claim 1 , wherein the directory is an empty directory, and further comprising, detecting a new node added to the directory, and in response to the detecting, associating the new node with the domain.

8 . A system, comprising:
a processor; and a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
maintaining domain membership information in association with a file object of a directory corresponding to a domain;
receiving a request to return the domain membership information of the file object;
based on the request, determining whether the domain membership information of the file object is up to date;
in response to the determining indicating that the domain membership information of the file object is up to date, returning the domain membership information; and
in response to the determining indicating that the domain membership information of the file object is not up to date, updating the domain membership information, the updating comprising obtaining updated domain membership information associated with a parent node of the file object, and returning the updated domain membership.

9 . The system of claim 8 , wherein the operations further comprise, creating the domain on the directory, associating the directory with the domain membership information, and associating the file object with the domain membership information.

10 . The system of claim 8 , wherein the maintaining the domain membership information in association with the file object comprises maintaining a domain identifier of the domain in metadata of the file object.

11 . The system of claim 8 , wherein the operations further comprise, detecting an operation that renames the directory above the file object to a namespace corresponding to a second domain, obtaining first domain membership data of the first domain and second domain membership data of the second domain, and, in response to the first domain membership data being determined to be different from the second membership data, updating the domain membership information of the file object based on the second membership data.

12 . The system of claim 8 , wherein the directory is a first directory, wherein the domain is a first domain, and wherein the operations further comprise, detecting an operation that provides a hard link the first directory to a second directory corresponding to a second domain, and updating the membership information associated with the file object based on domain data associated with the second domain.

13 . The system of claim 12 , wherein the operations further comprise, detecting an operation that removes the hard link between the first directory and the second directory, and updating the membership information associated with the file object based on the file object being determined to be no longer hard linked via the first directory to the second directory.

14 . The system of claim 8 , wherein the domain membership information of the file object comprises a first domain identifier and a second domain identifier, and wherein the operations further comprise, adding a third domain identifier to the domain membership information, in which the third identifier logically excludes the first domain identifier from the domain membership information.

15 . The system of claim 14 , wherein the returning the domain membership information comprises generating a response by excluding the first identifier from the response based on the third identifier, and returning the response comprising the second identifier.

16 . The system of claim 8 , wherein the domain membership information of the file object comprises a first domain identifier, and wherein the operations further comprise, maintaining the first domain identifier in a domain data structure in conjunction with data service-specific data.

17 . The system of claim 8 , wherein the determining whether the domain membership information of the file object is up to date comprises evaluating a domain membership information generation number associated with the domain membership information of the file object against a generation counter value corresponding to a most-recent domain creation.

18 . A machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
creating a domain in association with a directory in response to a first request from a data service, in response to the first request, associating a domain identifier with the directory, and associating nodes under the directory with the domain identifier, maintaining the domain identifier in a domain data structure entry, returning the domain identifier to the data service, receiving a second request from the data service to return domain membership information of a file object corresponding to one of the nodes under the directory, in response to the second request, determining the domain membership information of the file object, and returning the domain membership information, comprising the domain identifier, to the data service.

19 . The machine-readable storage medium of claim 18 , wherein the associating nodes under the directory with the domain identifier comprises tagging the nodes with the domain identifi

Assignees

• Emc Ip Holding Co Llc

Inventors

• Rajawat Mayank
• Naujok Barry
• Fahadullah Mohd

Classifications

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• H04L67/06

  specially adapted for file transfer, e.g. file transfer protocol [FTP] · CPC title

• G06F16/182 Primary

  Distributed file systems · CPC title

• G06F16/185

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F16/13 Primary

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title