Optimized file system layout for distributed consensus protocol

Patent metadata

Field	Value
Publication number	US-10229009-B2
Application number	US-201514971807-A
Country	US
Kind code	B2
Filing date	Dec 16, 2015
Priority date	Dec 16, 2015
Publication date	Mar 12, 2019
Grant date	Mar 12, 2019

Abstract

A layout of a file system is optimized to meet storage requirements of a distributed consensus protocol implemented on a plurality of nodes of a cluster. Illustratively, the file system has an on-disk layout representation that enables efficient input/output (I/O) operation performance. The on-disk layout of the file system embodies a plurality of file system objects: membership, snapshot and log objects. Each object is allocated an area or region (e.g., a contiguous storage space) on-disk. In addition, each object has two or more storage container instances, e.g., files. In the case of snapshot and membership objects, the two files of each object are used in a circular (alternating) fashion so that write operations directed to a first file storing a current copy of the snapshot/membership content may be performed without destroying a previous committed copy of the snapshot/membership content stored in a second file.

First claim

What is claimed is:

1. A system comprising:
a cluster of storage systems coupled to a network;
a memory coupled to a processor included in each storage system, the memory configured to store a file system executable by the processor that records configuration changes to the cluster via a distributed consensus protocol; and
a storage device connected to each storage system of the cluster and configured to store an on-disk layout of the file system having a plurality of file system objects, each file system object allocated a region on the storage device, each file system object further having two or more storage container instances, wherein the storage container instances for at least one object are used in an alternating arrangement so that write operations directed to a first storage container instance storing a current copy of content is performed without destroying a previous committed copy of the content stored in a second storage container instance.

2. The system of claim 1 wherein the storage container instances are files.

3. The system of claim 1 wherein the storage container instances are files and wherein each file contains a header that describes the file to enable the file system to perform a multi-block read operation to an end of the file instead of having to serially read all blocks using a multitude of operations.

4. The system of claim 1 wherein the plurality of file system objects include membership, snapshot and log objects.

5. The system of claim 1 wherein the plurality of file system object includes a log object written as a whole atomically.

6. The system of claim 1 wherein the at least one object is one of a snapshot and a membership objects, and wherein the content is one of snapshot and membership content.

7. The system of claim 1 wherein at least one file object is protected with an error correcting code that enables the distributed consensus protocol executed by the processor to detect and recover from partial write operations by falling back to the previous committed copy.

8. The system of claim 1 wherein a log region of the on-disk layout is configured to store sequential logs representing a state of the cluster as provided by the distributed consensus protocol.

9. The system of claim 1 wherein the distributed consensus protocol is one of Paxos and Raft.

10. The system of claim 1 wherein configuration updates for the distributed consensus protocol are received at the memory, organized as log records, and persistently stored to a log file of a log region as a sequential log.

11. The system of claim 1 wherein configuration updates for the distributed consensus protocol are stored as a compressed sequential log having a current state of the configuration updates as a snapshot representing a point-in-time image of the log.

12. The system of claim 1 wherein a current state of configuration changes are stored as a snapshot in a snapshot region of the on-disk layout.

13. The system of claim 12 wherein, in response to the current state of the configuration changes stored as a snapshot on the storage device, a storage space of a log file storing the configuration changes prior to the snapshot is reclaimed.

14. A method comprising:
connecting a storage array to a storage system of a cluster of storage systems, the storage system having a processor configured to store a file system executable by the processor to record configuration changes to the cluster via a distributed consensus protocol;
storing an on-disk layout of the file system on the storage array, the on-disk layout of the file system having a plurality of file system objects;
allocating a region on disk for each file system object, each file system object further having two or more files; and
employing the files for at least one object in an alternating arrangement so that write operations directed to a first file storing a current copy of content is performed without destroying a previous committed copy of the content stored in a second file.

15. The method of claim 14 further comprising:
describing each file with a header that enables the file system to perform a multi-block read operation to an end of the file instead of having to serially read all blocks using a multitude of operations.

16. The method of claim 14 wherein the plurality of file system objects include membership, snapshot and log objects.

17. The method of claim 14 further comprising:
protecting at least one file with an error correcting code that enables the distributed consensus protocol to detect and recover from partial write operations by falling back to the previous committed copy.

18. The method of claim 14 further comprising:
storing sequential logs in a log file of a log region of the on-disk layout, the sequential logs representing a state of the cluster as provided by the distributed consensus protocol.

19. The method of claim 14 wherein a log file storing configuration changes is rolled back with a single write operation through use of log entry information maintained in a memory of the storage system.

20. A non-transitory computer readable medium including program instructions for execution on a processor of a storage system of a cluster, the program instructions configured to:
record changes to a cluster configuration via a distributed consensus protocol;
store an on-disk layout of a file system on disks of a storage array connected to the storage system, the on-disk layout of the file system having a plurality of file system objects;
allocate a region on disk for each file system object having two or more files; and
employ the files for at least one object in an alternating arrangement so that write operations directed to a first file storing a current copy of the cluster configuration is performed without destroying a previous committed copy of the cluster configuration stored in a second file.

Assignees

• Netapp Inc

Inventors

• Muth John
• Gomez Juan C

Classifications

• G06F11/1471

  involving logging of persistent data for recovery · CPC title

• G06F3/06

  Digital input from, or digital output to, record carriers {, e.g. RAID, emulated record carriers or networked record carriers} · CPC title

• G06F11/1464 Primary

  for networked environments · CPC title

• G06F3/0688

  Non-volatile semiconductor memory arrays · CPC title

• G06F11/1076

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title