Storing compressed and uncompressed data in blocks having different allocation unit sizes

Patent metadata

Field	Value
Publication number	US-9880928-B1
Application number	US-201414497814-A
Country	US
Kind code	B1
Filing date	Sep 26, 2014
Priority date	Sep 26, 2014
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

Improved techniques for storing data involve storing compressed data in blocks of a first AU size and storing uncompressed data in blocks of a second AU size larger than the first AU size. For example, when a storage processor compresses a chunk of data, the storage processor checks whether the compressed chunk fits in the smaller AU size. If the compressed chunk fits, then the storage processor stores a compressed chunk in a block having the smaller AU size. Otherwise, the storage processor stores the uncompressed chunk in a block having the larger AU size. Advantageously, the improved techniques promote better disk and cache utilization, which improves performance without disrupting block mapping.

First claim

What is claimed is:

1. A method of processing write requests in a data storage system, the method comprising:
receiving, by the data storage system over a network, write requests from a set of hosts, the write requests specifying data to be written to non-volatile storage;
storing a first portion of the data in compressed form in blocks of non-volatile storage having a first allocation unit size; and
storing a second portion of the data in uncompressed form in blocks of non-volatile storage having a second allocation unit size, the second allocation unit size being larger than the first allocation unit size,
wherein a set of the blocks of the non-volatile storage store data for realizing a file system, the file system operating with a logical block size equal to the second allocation unit size; and
wherein the method further comprises:
receiving a request to read a block of the non-volatile storage having the first allocation unit size; and
performing a decompression operation on compressed data stored in the block to produce an amount of uncompressed data having a size equal to the second allocation unit size,

wherein the file system has a leaf indirect block including a set of pointers to blocks storing data of the file system,
wherein the leaf indirect block further includes a first bit for each of the set of pointers, a value of the first bit indicating whether that pointer points to a block having the first allocation unit size, and
wherein the leaf indirect block of the file further includes a second bit for each of the set of pointers, the value of the second bit indicating whether that pointer points to a block storing data on which a compression operation was attempted and failed.

2. A method as in claim 1 , further comprising:
upon receiving each of the write requests, caching the data specified by that write request in a volatile memory of the data storage system; and
after caching the data, acknowledging to the set of hosts that each of the write requests was fulfilled.

3. A method as in claim 2 , further comprising mirroring the data specified by each of the write requests to a second volatile memory, the volatile memory and the second volatile memory having battery backup; and
wherein acknowledging that each write request was fulfilled occurs after the data specified by that write request are cached in the volatile memory and mirrored to the second volatile memory.

4. A method as in claim 2 , further comprising, after caching the data specified by each write request in the volatile memory:
aggregating the data specified by that write request in chunks in the cache, each of the chunks having a size equal to the second allocation unit size; and
performing a decision operation to determine whether to store each chunk in compressed form in the blocks of the non-volatile storage having the first allocation unit size.

5. A method as in claim 4 , wherein performing the decision operation includes:
performing a compression operation on each of a set of the chunks to form respective compressed chunks; and
for each compressed chunk resulting from the compression operation, performing a testing operation to produce a testing result indicating whether the compressed chunk fits in a block having the first allocation unit size; and
wherein the method further comprises:
flushing each compressed chunk that fits in a block having the first allocation unit size to a block having the first allocation unit size; and
flushing each chunk whose respective compressed chunk does not fit in a block having the first allocation unit size to a block having the second allocation unit size.

6. A method as in claim 5 , wherein performing the decision operation further includes:
monitoring a recent history of testing results for a second set of the chunks; and
in response to the recent history indicating that recently a sufficient number of the second set of the chunks have not been able to fit into blocks having the first allocation unit size, flushing subsequent chunks to blocks having the second allocation unit size without performing any compression operation.

7. A method as in claim 6 , wherein performing the decision operation further includes:
after the occurrence of a predetermined event, resuming the compression operation on subsequent chunks to produce respective, subsequent compressed chunks; and
for each subsequent compressed chunk resulting from the compression operation, performing the testing operation to produce a testing result indicating whether the subsequent compressed chunk fits in a block having the first allocation unit size; and
wherein the method further comprises:
flushing each subsequent compressed chunk that fits in a block having the first allocation unit size to a block having the first allocation unit size; and
flushing each subsequent chunk whose respective, subsequent compressed chunk does not fit in a block having the first allocation unit size to a block having the second allocation unit size.

8. A method as in claim 5 , wherein the volatile memory has a high water mark indicating a high level of utilization of the volatile memory for caching data, and a low water mark below the high water mark; and
wherein performing the decision operation includes:
examining whether the high water mark of the volatile memory has been obtained;
in response to the high water mark being obtained, storing a third set of the chunks in uncompressed form in a block of the non-volatile storage having the second allocation unit size without performing any compression operation until a subsequent examination result indicates that the low water mark has been obtained;

in response to the low water mark being subsequently obtained, resuming the compression operation on subsequent chunks to produce respective, subsequent compressed chunks; and
for each subsequent compressed chunk resulting from the compression operation, performing the testing operation to produce a testing result indicating whether the subsequent compressed chunk fits in a block having the first allocation unit size; and
wherein the method further comprises:
flushing each subsequent compressed chunk that fits in a block having the first allocation unit size to a block having the first allocation unit size; and
flushing each subsequent chunk whose respective, subsequent compressed chunk does not fit in a block having the first allocation unit size to a block having the second allocation unit size.

9. A method as in claim 1 ,
wherein the method further comprises:
reading the value of the first bit for a pointer after receiving a request to read the block to which that pointer points;
performing the decompression operation on data stored in the block when the first bit has a first value.

10. A method as in claim 9 ,
wherein the method further comprises:
for each of the set of pointers pointing to a block having the second allocation unit size, reading the value of the second bit for that pointer after receiving a request to read the block to which that pointer points.

11. A method as in claim 1 , wherein the method further comprises realizing one of a LUN, vVol, and HFS in the form of a file.

12. A method as in claim 1 , further comprising performing a deduplication operation on a duplicate block of the file system, the deduplication operation including:
reclaiming a duplicate block having the first allocation unit size from the file system; and
reclaiming a duplicate block having the second allocation unit size from the file system.

13. A method as in claim 1 , further comprising performing a snapshot operation on a file stored in the

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Bono Jean-Pierre
• Armangau Philippe

Classifications

• G06F3/0608 Primary

  Saving storage space on storage systems · CPC title

• G06F12/0246 Primary

  in block erasable memory, e.g. flash memory · CPC title

• G06F2212/7207

  management of metadata or control data · CPC title

• G06F2212/7202

  Allocation control and policies · CPC title

• G06F2212/2228

  Battery-backed RAM · CPC title