Managing file system metadata using persistent cache

Patent metadata

Field	Value
Publication number	US-11068299-B1
Application number	US-201715669364-A
Country	US
Kind code	B1
Filing date	Aug 4, 2017
Priority date	Aug 4, 2017
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

A technique for managing metadata in a data storage system includes receiving a set of data to be stored in a file system of the data storage system and calculating values of metadata blocks that the file system will use to organize the set of data in the file system. The technique aggregates values of the metadata blocks and sends them in a single, atomic transaction to a persistent cache. The transaction either succeeds, in which case the persistent cache stores the values of all of the metadata blocks, or it fails, in which case the persistent cache stores none of the values of the metadata blocks. Over time, the persistent cache flushes the values of the metadata blocks to a set of non-volatile storage devices that back the metadata blocks in the data storage system.

First claim

What is claimed is:

1. A method of managing metadata in a data storage system, the method comprising:
receiving an I/O (input/output) request that specifies a set of data to be written to a file system in the data storage system, the file system backed by a set of non-volatile storage devices;
computing values of multiple metadata blocks that the file system will use to organize the set of data in the file system, the multiple metadata blocks including at least one of a virtual block map (VBM), an allocation bitmap, and a superblock;
aggregating the computed values of the metadata blocks into a single transaction; and
atomically issuing the transaction to a persistent cache, such that values of all of the metadata blocks are written to the persistent cache or none of them are, the persistent cache thereafter flushing the values of the metadata blocks, or updated versions thereof, to the set of non-volatile storage devices backing the file system,
wherein the persistent cache provides multiple flushing policies, and wherein the method further comprises specifying a delayed flushing policy for pages in the persistent cache that store the values of the multiple metadata blocks, the delayed flushing policy specifying flushing at a slower rate than another flushing policy provided by the persistent cache,
wherein the method further comprises:
receiving a second I/O request specifying a second set of data to be written to the file system;
computing values of a second set of metadata blocks that the file system will use to organize the second set of data in the file system, the second set of metadata blocks including a common metadata block that that is also one of the multiple metadata blocks, a value of the common metadata block stored in a cache page of the persistent cache; and
atomically issuing a second transaction to the persistent cache, such that the values of all of the second set of metadata blocks are written to the persistent cache or none of them are, the persistent cache updating the cache page that stores the value of the common metadata block to reflect a change in the value of common metadata block for incorporating the second set of data,

wherein the data storage system further includes a set of internal volumes operatively disposed between the persistent cache and the set of non-volatile storage devices, and wherein writing the transaction to the persistent cache includes writing the values of the multiple metadata blocks to pages of the persistent cache using an addressing scheme that addresses pages by identifier of one of the set of internal volumes and offset into that internal volume.

2. The method of claim 1 , wherein, when atomically issuing the transaction to the persistent cache, the method further comprises specifying the set of data as part of the transaction, such that both the set of data and all of the metadata blocks are written to the persistent cache or none of them are.

3. The method of claim 1 , wherein the persistent cache provides multiple flushing policies, and wherein the method further comprises specifying a delayed flushing policy for pages in the persistent cache that store the values of the multiple metadata blocks, the delayed flushing policy specifying flushing at a slower rate than another flushing policy provided by the persistent cache.

4. The method of claim 1 , wherein atomically issuing the transaction includes generating a single IRP (I/O request packet) that specifies the computed values of the multiple metadata blocks.

5. The method of claim 4 , wherein the single IRP includes, for each of the multiple metadata blocks, one of (i) a computed value of the metadata block or (ii) a reference to a location where that metadata block value can be found.

6. The method of claim 5 , wherein, the single IRP further includes a reference to the set of data, such that both the set of data and the values of all of the metadata blocks are atomically written to the persistent cache or none of them are.

7. The method of claim 5 , wherein atomically writing the second transaction includes generating a second IRP that specifies the computed values of the second set of metadata blocks, the second IRP including a reference to the cache page in the persistent cache that stores the common metadata block.

8. The method of claim 7 , further comprising:
receiving an acknowledgment from the persistent cache that the set of data and the values of all of the multiple metadata blocks have been stored in the persistent cache; and
upon receiving the acknowledgement from the persistent cache, sending an acknowledgement to a requestor of the I/O request that the I/O request has been completed.

9. The method of claim 1 , wherein atomically issuing the transaction includes generating a single IRP (I/O request packet) that specifies the computed values of the multiple metadata blocks.

10. The method of claim 1 , further comprising updating a particular metadata block of the metadata blocks in the persistent cache after atomically issuing the transaction to the persistent cache, and later flushing the updated particular metadata block to the set of non-volatile storage devices for storage therein.

11. A data storage apparatus, comprising a set of processing units and memory, the memory coupled to the set of processing units and storing executable instructions which, when executed by the set of processing units, cause the set of processing units to:
receive an I/O (input/output) request that specifies a set of data to be written to a file system in the data storage system, the file system backed by a set of non-volatile storage devices;
compute values of multiple metadata blocks that the file system will use to organize the set of data in the file system, the multiple metadata blocks including at least one of a virtual block map (VBM), an allocation bitmap, and a superblock;
aggregate the computed values of the metadata blocks into a single transaction; and
atomically issue the transaction to a persistent cache, such that values of all of the metadata blocks are written to the persistent cache or none of them are, the persistent cache thereafter flushing the values of the metadata blocks, or updated versions thereof, to the set of non-volatile storage devices backing the file system for storage of the all of the metadata blocks, or updated versions thereof, in the set of non-volatile storage devices,
wherein the persistent cache provides multiple flushing policies, and wherein the executable instructions further cause the set of processing units to specify a delayed flushing policy for pages in the persistent cache that store the values of the multiple metadata blocks, the delayed flushing policy specifying flushing at a slower rate than another flushing policy provided by the persistent cache,
wherein the executable instructions further cause the set of processing units to:
receive a second I/O request specifying a second set of data to be written to the file system;
compute values of a second set of metadata blocks that the file system will use to organize the second set of data in the file system, the second set of metadata blocks including a common metadata block that that is also one of the multiple metadata blocks, a value of the common metadata block stored in a cache page of the persistent cache; and
atomically issue a second transaction to the persistent cache, such that the values of all of the second set of metadata blocks are written to the persistent cache or none of them are, the persistent cache updating the cache page that stores the value of the common metadata block to reflect a change in the value of common metadata block for incorporating the second set of

Assignees

• Emc Ip Holding Co Llc

Inventors

• Armangau Philippe
• Seibel Christopher A
• Gillono John
• Caram Bruce E
• Wang Yubing
• Bono Jean-Pierre

Classifications

• G06F3/0656

  Data buffering arrangements · CPC title

• G06F3/0643

  Management of files · CPC title

• G06F3/064

  Management of blocks · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/061

  Improving I/O performance · CPC title