Method and apparatus for compressing metadata in a file system

Patent metadata

Field	Value
Publication number	US-10901949-B2
Application number	US-201715628563-A
Country	US
Kind code	B2
Filing date	Jun 20, 2017
Priority date	Jun 21, 2016
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

Embodiments of the present disclosure relate to a method and an apparatus for compressing metadata in a file system. The method comprises, in response to receiving a first request for writing first data to a file, determining whether the first request is for an initial write to a storage area associated with a second indirect block in the first group of indirect blocks, the first group of indirect blocks at least including a first indirect block and the second indirect block. The method further comprises, in response to the initial write, allocating a first group of data blocks for writing the first data on a storage device. In addition, the method further comprises compressing the first group of indirect blocks by encoding a first group of storage addresses corresponding to the first group of data blocks into the first indirect block.

First claim

We claim:

1. A method for compressing metadata in a file system,
the metadata comprising a first group of indirect blocks, the method comprising:
in response to receiving a first request for writing first data to a file, wherein the first request includes a first offset indicating a start location in the file to which the first data is to be written and a first length of the first data, determining, based on the first offset and the first length, whether the first request is for an initial write to a storage area associated with a second indirect block in the first group of indirect blocks, the first group of indirect blocks at least including a first indirect block and the second indirect block, and wherein the first indirect block includes a first area for compressing continuous storage addresses, the first area including a plurality of first sub-areas, each of the plurality of first sub-areas corresponding to a same number of data blocks;
in response to the initial write, allocating a first group of data blocks for writing the first data on a storage device;
compressing the first group of indirect blocks by encoding a first group of storage addresses corresponding to the first group of data blocks into the first indirect block, wherein compressing the first group of indirect blocks causes the first indirect block to store encodings of storage addresses corresponding to data blocks on the storage device that are allocated for writing data to storage areas of the file associated with multiple ones of the indirect blocks in the first group of indirect blocks; and
wherein encoding the first group of storage addresses includes:

determining, based on the first offset and the first length, at least one first sub-area associated with the first group of data blocks, the at least one first sub-area being included in the plurality of first sub-areas in the first indirect block, wherein the at least one first sub-area includes a first field indicating a storage address of a starting data block in the same number of data blocks and a first bitmap indicating whether a respective storage address of each of the same number of data blocks is valid, encoding the first group of storage addresses into the at least one first sub-area,
determining, based on the second indirect block, the first indirect block in the first group of indirect blocks, updating the first field based on the first group of storage addresses, and in response to determining that part of the first group of storage addresses are discontinuous, setting at least one bit corresponding to the part of the first group of storage addresses in the first bitmap to be invalid.

2. The method of claim 1 , wherein determining whether the first request is for an initial write to a storage area associated with a second indirect block further comprises:
determining, based on the first offset and the first length, a first intermediate indirect block associated with the first data, the first intermediate indirect block recording at least one address of the first group of indirect blocks;
determining, based on the first offset and the first length, that the first data is to be written to the storage area associated with the second indirect block; and
in response to determining that an address of the second indirect block as recorded in the first intermediate indirect block is invalid, determining that the first request is for the initial write to the storage area.

3. The method of claim 1 , wherein the first indirect block further includes a second area for recording discontinuous storage addresses, and encoding the first group of storage addresses further comprises:
recording the part of the first group of storage addresses in the second area.

4. The method of claim 2 , further comprising:
after performing the initial write, updating, in the first intermediate indirect block, the address of the second indirect block with an address of the first indirect block such that the second indirect block points to the first indirect block.

5. The method of claim 1 , further comprising:
in response to determining that space occupied in the first indirect block exceeds a predefined threshold, dividing the first group of indirect blocks into at least two groups of indirect blocks, the at least two groups of indirect blocks including a second group of indirect blocks and a third group of indirect blocks, the second group of indirect blocks including the first indirect block and the third group of indirect blocks at least including a third indirect block; and
encoding a storage address as encoded in the first indirect block into the third indirect block, the storage address being associated with one of the third group of indirect blocks.

6. The method of claim 5 , further comprising:
in response to receiving a second request for reading second data from a file, determining whether the second data is to be read from a storage area associated with a fifth indirect block in a fourth group of indirect blocks at least including a fourth indirect block and the fifth indirect block; and
in response to determining that the second data is to be read from the storage area, determining from the fourth indirect block a second group of storage addresses of a second group of data blocks for storing the second data.

7. The method of claim 6 , wherein the second request includes a second offset indicating a start location in the file to which the second data is written, and a second length of the second data, and the determining whether the second data is to be read from a storage area associated with a fifth indirect block comprises:
determining, based on the second offset and the second length, a second intermediate indirect block associated with the second data, the second intermediate indirect block recording at least one address of the fourth group of indirect blocks; and
determining, based on the second offset and the second length, that the second data is to be read from the storage area associated with the fifth indirect block.

8. The method of claim 6 , wherein the determining a second group of storage addresses comprises:
determining, based on the fifth indirect block, the fourth indirect block in the fourth group of indirect blocks.

9. The method of claim 7 , wherein the fourth indirect block includes a third area for compressing continuous storage addresses, the third area including a plurality of third sub-areas, each of the plurality of third sub-areas corresponding to a same number of data blocks, and the determining a second group of storage addresses comprises:
determining, based on the second offset and the second length, at least one third sub-area associated with the second group of data blocks, the at least one third sub-area being included in the plurality of third sub-areas; and
determining the second group of storage addresses from the at least one third sub-area.

10. The method of claim 9 , wherein the at least one third sub-area includes a second field indicating a storage address of a start data block in the same number of data blocks and a second bitmap indicating whether a respective storage address of each of the same number of data blocks is valid, and the determining the second group of storage addresses from the at least one third sub-area comprises:
determining, based on the second field and a valid bit contained in the second bitmap, a first storage address corresponding to the valid bit in the second group of storage addresses.

11. The method of claim 10 , wherein the fourth indirect block further includes a fourth area for recording discontinuous storage addresses, and the determining a second group of storage addresses comprises:

Assignees

• Emc Ip Holding Co Llc

Inventors

• Zhao Junping Frank
• Zhao Accela Yilong

Classifications

• H03M7/3091

  Data deduplication · CPC title

• H03M7/707

  Structured documents, e.g. XML · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title

• G06F3/0643

  Management of files · CPC title