Parallel file system with metadata distributed across partitioned key-value store

Patent metadata

Field	Value
Publication number	US-9767107-B1
Application number	US-201313931832-A
Country	US
Kind code	B1
Filing date	Jun 29, 2013
Priority date	Jun 29, 2013
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

Improved techniques are provided for storing metadata associated with a plurality of sub-files associated with a single shared file in a parallel file system. The shared file is generated by a plurality of applications executing on a plurality of compute nodes. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file generated by an application executing on the compute node and metadata for the at least one portion of the shared file on one or more object storage servers. The compute node is also configured to implement a partitioned data store for storing a partition of the metadata for the shared file, wherein the partitioned data store communicates with partitioned data stores on other compute nodes using a message passing interface. The partitioned data store can be implemented, for example, using Multidimensional Data Hashing Indexing Middleware (MDHIM).

First claim

What is claimed is:

1. An apparatus comprising:
a compute node of a parallel file system;
the compute node being configured to communicate with a plurality of object storage servers and with a plurality of other compute nodes over a network, wherein a plurality of applications executing on said plurality of compute nodes generate a shared file;
the compute node further configured to implement a Parallel Log Structured File System (PLFS) library to store at least one portion of said shared file generated by an application executing on said compute node and PLFS metadata for said at least one portion of said shared file on one or more of said plurality of object storage servers, wherein said PLFS metadata for said at least one portion of said shared file comprises a key-value pair having a key based on a logical offset of said at least one portion of said shared file;
the compute node further configured to implement a partitioned key-value data store for storing a partition of said PLFS metadata for said shared file, wherein said partitioned key-value data store communicates with partitioned key-value data stores on said other compute nodes using a message passing interface to cooperatively store the PLFS metadata, wherein said partition comprises a given sub-range of a key space for said PLFS metadata, wherein said key-value pair for said PLFS metadata for said at least one portion of said shared file is inserted into a corresponding one of said partitions stored by a respective partitioned key-value data store as a new key-value record based on said key when said at least one portion is written to said shared file; and
the compute node being implemented utilizing at least one processing device coupled to a memory.

2. The apparatus of claim 1 wherein said PLFS library communicates with said partitioned key-value data store using function calls.

3. The apparatus of claim 1 wherein said PLFS metadata for said shared file is sharded across said partitioned key-value data store.

4. The apparatus of claim 1 wherein each of the object storage servers is configured to interface with at least one object storage target.

5. The apparatus of claim 1 wherein the partitioned key-value data store is configured to process one or more metadata requests when one or more of said plurality of applications executing on said compute nodes performs a read operation on said shared file.

6. The apparatus of claim 1 wherein the partitioned key-value data store is implemented using Multidimensional Data Hashing Indexing Middleware (MDHIM).

7. The apparatus of claim 6 wherein one or more MDHIM clients run on each of said compute nodes and communicate with a plurality of MDHIM servers in said parallel file system.

8. The apparatus of claim 7 wherein each of said plurality of MDHIM servers store a given partition of said PLFS metadata for said shared file comprising said sub-range of said key space for said PLFS metadata for said shared file.

9. The apparatus of claim 8 wherein a given MDHIM server storing a particular sub-range of said PLFS metadata is contacted to read or write PLFS metadata within said sub-range.

10. The apparatus of claim 1 wherein the parallel file system comprises a Lustre file system.

11. A processing platform comprising a plurality of processing devices and the apparatus of claim 1 .

12. A method comprising:
configuring a compute node of a parallel file system to communicate with a plurality of object storage servers and with a plurality of other compute nodes over a network, wherein a plurality of applications executing on said plurality of compute nodes generate a shared file; and
configuring the compute node to implement a Parallel Log Structured File System (PLFS) library to store at least one portion of said shared file generated by an application executing on said compute node and PLFS metadata for said at least one portion of said shared file on one or more of said plurality of object storage servers, wherein said PLFS metadata for said at least one portion of said shared file comprises a key-value pair having a key based on a logical offset of said at least one portion of said shared file; and
configuring the compute node to implement a partitioned key-value data store for storing a partition of said PLFS metadata for said shared file, wherein said partitioned key-value data store communicates with partitioned key-value data stores on said other compute nodes using a message passing interface to cooperatively store the PLFS metadata, wherein said partition comprises a given sub-range of a key space for said PLFS metadata, wherein said key-value pair for said PLFS metadata for said at least one portion of said shared file is inserted into a corresponding one of said partitions stored by a respective partitioned key-value data store as a new key-value record based on said key when said at least one portion is written to said shared file;
the compute node being implemented utilizing at least one processing device coupled to a memory.

13. The method of claim 12 wherein said PLFS library communicates with said partitioned key-value data store using function calls.

14. The method of claim 12 wherein said PLFS metadata for said shared file is sharded across said partitioned key-value data store.

15. The method of claim 12 further comprising the step of configuring each of the object storage servers to interface with at least one object storage target.

16. The method of claim 12 further comprising the step of configuring the partitioned key-value data store to process one or more metadata requests when one or more of said plurality of applications executing on said compute nodes performs a read operation on said shared file.

17. The method of claim 12 wherein the partitioned key-value data store is implemented using Multidimensional Data Hashing Indexing Middleware (MDHIM).

18. The method of claim 17 wherein one or more MDHIM clients run on each of said compute nodes and communicate with a plurality of MDHIM servers in said parallel file system.

19. The method of claim 18 wherein each of said plurality of MDHIM servers store a given partition of said PLFS metadata for said shared file comprising said sub-range of said key space for said PLFS metadata for said shared file.

20. The method of claim 19 wherein a given MDHIM server storing a particular sub-range of said PLFS metadata is contacted to read or write PLFS metadata within said sub-range.

21. A computer program product comprising a processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed perform the steps of the method of claim 12 .

22. A parallel file system comprising:
a plurality of compute nodes; and
a plurality of object storage servers;
one of the plurality of compute nodes being configured to communicate with said plurality of object storage servers and with others of said plurality of compute nodes over a network, wherein a plurality of applications executing on said plurality of compute nodes generate a shared file;
the one compute node further configured to implement a Parallel Log Structured File System (PLFS) library to store at least one portion of said shared file generated by an application executing on said one compute node and PLFS metadata for said at least one portion of said shared file on one or more of said plurality of object storage servers, wherein said PLFS metadata for said at le

Assignees

• Emc Ip Holding Co Llc
• Los Alamos Nat Security Llc

Inventors

• Bent John M
• Faibish Sorin
• Grider Gary
• Torres Aaron

Classifications

• G06F17/30094 Primary

  Physics · mapped topic

• G06F16/1858

  Parallel file systems, i.e. file systems supporting multiple processors · CPC title

• G06F16/134 Primary

  Distributed indices · CPC title