Compressing metadata blocks prior to writing the metadata blocks out to secondary storage

Patent metadata

Field	Value
Publication number	US-9880762-B1
Application number	US-201514984063-A
Country	US
Kind code	B1
Filing date	Dec 30, 2015
Priority date	Dec 30, 2015
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

A technique manages file system metadata and is performed in a data storage system that maintains a file system on behalf of a set of host computers. The technique involves modifying a first metadata block of the file system in response to a file system change, the first metadata block containing file system metadata. The technique further involves performing a compression operation to generate a compressed metadata block in volatile storage from the first metadata block, the compressed metadata block having a size that is smaller than a size of the first metadata block. The technique further involves writing the compressed metadata block from the volatile storage (volatile cache memory) to non-volatile storage (e.g., flash memory).

First claim

What is claimed is:

1. In a data storage system that maintains a file system on behalf of a set of host computers, a method of managing file system metadata, the method comprising:
modifying file system data of the file system in response to a file system change;
providing an initial metadata block containing a copy of the modified file system data;
performing a compression operation on the initial metadata block to generate a compressed metadata block in volatile storage, the compressed metadata block having a size that is smaller than a size of the initial metadata block;
writing the compressed metadata block from the volatile storage to non-volatile storage;
reading the compressed metadata block from the non-volatile storage;
performing a decompression operation on the compressed metadata block to generate a decompressed metadata block, the decompressed metadata block having a size that is larger than the size of the compressed metadata block;
performing one or more of an error detection operation and an error correction operation while performing one or more of the compression operation and the decompression operation to obtain confirmation results pertaining to correct decompression of the compressed metadata block; and
verifying that contents of the decompressed metadata block are valid based on the confirmation results, thereby avoiding storage of a checksum for validating the contents of the decompressed metadata block.

2. A method as in claim 1 further comprising:
entering the initial metadata block into a metadata transaction log from which dirty metadata blocks are periodically flushed to the non-volatile storage.

3. A method as in claim 2 wherein performing the compression operation to generate the compressed metadata block in the volatile storage from the initial metadata block includes:
as part of a flush operation which flushes dirty metadata blocks of the metadata transaction log to the non-volatile storage, applying a compression algorithm to the initial metadata block to form the compressed metadata block.

4. A method as in claim 1 wherein the file system data includes an inode structure,
wherein the modifying of the file system data includes modifying the inode structure in response to the file system change, and
wherein the providing of the initial metadata block includes providing the initial metadata block containing a copy of the modified inode structure.

5. A method as in claim 4 wherein the performing of the compression operation includes performing the compression operation on the copy of the modified inode structure, the compressed metadata block including a compressed copy of the modified inode structure, and
wherein the performing of the decompression operation includes performing the decompression operation on the compressed copy of the modified inode structure.

6. In a data storage system that maintains a file system on behalf of a set of host computers, a method of managing file system metadata, the method comprising:
modifying a first metadata block of the file system in response to a file system change, the first metadata block containing file system metadata;
performing a compression operation to generate a compressed metadata block in volatile storage from the first metadata block, the compressed metadata block having a size that is smaller than a size of the first metadata block;
writing the compressed metadata block from the volatile storage to non-volatile storage;
reading the compressed metadata block from the non-volatile storage;
performing a decompression operation to generate a decompressed metadata block from the compressed metadata block, the decompressed metadata block having a size that is larger than the size of the compressed metadata block; and
analyzing a result of the decompression operation to verify that contents of the decompressed metadata block are valid,
wherein modifying the first metadata block includes:
entering the first metadata block into a metadata transaction log from which dirty metadata blocks are periodically flushed to the non-volatile storage,

wherein performing the compression operation to generate the compressed metadata block in the volatile storage from the first metadata block includes:
as part of a flush operation which flushes dirty metadata blocks of the metadata transaction log to the non-volatile storage, applying a compression algorithm to the first metadata block to form the compressed metadata block,

wherein the first metadata block is an indirect block (IB) of an inode structure of the file system,
wherein a respective metadata block (BMD) contains additional metadata associated with the IB, and
wherein entering the first metadata block into the metadata transaction log includes adding the IB to the metadata transaction log without updating the BMD and without adding the BMD to the metadata transaction log.

7. A method as in claim 6 wherein the IB is 8 KB in size; and
wherein the compressed metadata block is 4 KB in size.

8. A method as in claim 6 wherein the flush operation (i) coalesces the IB with other compressed metadata blocks into an extent for synchronization to flash memory and (ii) writes the extent to the flash memory.

9. A method as in claim 6 wherein the metadata transaction log resides in first volatile cache memory; and wherein adding the IB to the metadata transaction log includes:
placing the IB in the metadata transaction log residing in the first volatile cache memory and placing a copy of the IB in second volatile cache memory which mirrors the first volatile cache memory.

10. A method as in claim 6 wherein writing the compressed metadata block from the volatile storage to the non-volatile storage includes writing a particular extent to the non-volatile storage, the extent including the IB; and
wherein the method further comprises:
in response to another file system change, reading a group of extents from the non-volatile storage, the group of extents including the particular extent, and
performing an XOR operation on the group of extents to determine whether the IB was involved in a lost write operation.

11. Data storage equipment that maintains a file system on behalf of a set of host computers, the data storage equipment comprising:
a communications interface which is constructed and arrange to communicate with the set of host computers;
memory which includes volatile storage and non-volatile storage; and
control circuitry coupled to the communications interface and the memory, the memory storing instructions which, when carried out by the control circuitry, cause the control circuitry to:
modify file system data of the file system in response to a file system change;
provide an initial metadata block containing a copy of the modified file system data,
perform a compression operation on the initial metadata block to generate a compressed metadata block in the volatile storage, the compressed metadata block having a size that is smaller than a size of the initial metadata block;
write the compressed metadata block from the volatile storage to the non-volatile storage;
read the compressed metadata block from the non-volatile storage;
perform a decompression operation on the compressed metadata block to generate a decompressed metadata block, the decompressed metadata block having a size that is larger than the size of the compressed metadata block;
perform one or more of an error detection operation and an error correction operation while performing one or more of the compression operation and the decompression operation to obtain confirmation results pertaining to correct decompression of the compressed metadata block; and
v

Assignees

• Emc Ip Holding Co Llc

Inventors

• Armangau Philippe
• Seibel Christopher A

Classifications

• G06F3/0619 Primary

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0643

  Management of files · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title