Combining hardware and software approaches for inline data compression

Patent metadata

Field	Value
Publication number	US-9985649-B1
Application number	US-201615197028-A
Country	US
Kind code	B1
Filing date	Jun 29, 2016
Priority date	Jun 29, 2016
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A technique for managing data storage applies both inline software compression and inline hardware compression in a data storage system, using both types of compression together. The data storage system applies inline software compression for compressing a first set of newly arriving data and applies inline hardware compression for compressing a second set of newly arriving data. Both sets of data are directed to a data object, and the data storage system compresses both sets of data without first storing uncompressed versions thereof in the data object.

First claim

What is claimed is:

1. A method of managing data storage, the method comprising:
receiving multiple requests to write data to a data object stored in a data storage system, the data storage system including a set of processing units and memory;
performing a first inline compression of a first set of the data using software compression, the software compression carried out by executing a set of instructions stored in the memory by the set of processing units, the first inline compression producing a first set of compressed data and performed without storing the first set of data in the data object prior to compression;
performing a second inline compression of a second set of the data using hardware compression, the hardware compression carried out by compression hardware running in the data storage system, the second inline compression producing a second set of compressed data and performed without storing the second set of data in the data object prior to compression;
storing the first set of compressed data in the data object; and
storing the second set of compressed data in the data object.

2. The method of claim 1 , further comprising operating a compression policy in the data storage system, the compression policy configured to select between software compression and hardware compression for compressing newly arriving data, based on a set of operating conditions in the data storage system.

3. The method of claim 2 , wherein the compression policy is initially configured to direct a first share of newly arriving data writes to software compression and to direct a second share of newly arriving data writes to hardware compression, the first share and the second share initially being predetermined.

4. The method of claim 3 , wherein the compression policy is further configured to vary at least one of the first share and the second share over time as the set of operating conditions in the data storage system changes.

5. The method of claim 4 , further comprising monitoring a number of threads that are performing inline compression on the set of processing units, wherein the set of operating conditions includes the monitored number of threads.

6. The method of claim 5 , wherein the compression policy is further configured to increase the second share of newly arriving data writes directed to hardware compression in response to an increase in the monitored number of threads that are performing inline compression.

7. The method of claim 5 , further comprising monitoring a utilization level of the compression hardware that perform inline compression, wherein the set of operating conditions further includes the monitored utilization level of the compression hardware.

8. The method of claim 7 , wherein the compression policy is further configured to decrease the second share of newly arriving data writes directed to hardware compression in response to an increase in the monitored utilization level.

9. The method of claim 5 , wherein the data storage system further includes a cache configured to receive data specified in newly arriving data writes, and wherein compression policy is further configured to increase the second share of newly arriving data writes in response to the cache becoming full.

10. The method of claim 5 , further comprising decompressing at least some of the second set of compressed data, which was compressed by hardware compression, using software decompression.

11. The method of claim 10 , further comprising operating a decompression policy in the data storage system, the decompression policy configured to respond to requests for data reads by selecting between software decompression and hardware decompression based on the set of operating conditions in the data storage system.

12. The method of claim 5 ,
wherein storing the first set of compressed data includes storing a first header with the first set of compressed data, the first header indicating a first compression method used to compress the first set of compressed data, and
wherein storing the second set of compressed data includes storing a second header with the second set of compressed data, the second header indicating a second compression method used to compress the second set of compressed data.

13. The method of claim 12 , further comprising:
decompressing the first set of compressed data by (i) reading the first header, (ii) selecting a first decompression method based on the first compression method indicated in the first header, and (iii) applying the selected first decompression method to decompress the first set of compressed data; and
decompressing the second set of compressed data by (i) reading the second header, (ii) selecting a second decompression method based on the second compression method indicated in the second header, and (iii) applying the selected second decompression method to decompress the second set of compressed data.

14. The method of claim 5 , wherein the data storage system stores data in multiple storage extents, and wherein the method further includes:
monitoring a data temperature of each of first and second storage extents, the data temperature of each of the first and second storage extents indicating a respective level of read and/or write activity on that storage extent;
in response to the first storage extent having a relatively high data temperature, selecting, by the compression policy, a compression method that produces a lower compression ratio when compressing data directed to the first storage extent; and
in response to the second storage extent having a relatively low data temperature, selecting, by the compression policy, a compression method that produces a higher compression ratio when compressing data directed to the second storage extent.

15. A data storage system, comprising control circuitry that includes a set of processing units coupled to memory, the control circuitry constructed and arranged to:
receive multiple requests to write data to a data object stored in a data storage system, the data storage system including a set of processing units and memory;
perform a first inline compression of a first set of the data using software compression, the software compression carried out by executing a set of instructions stored in the memory by the set of processing units, the first inline compression producing a first set of compressed data and performed without storing the first set of data in the data object prior to compression;
perform a second inline compression of a second set of the data using hardware compression, the hardware compression carried out by compression hardware running in the data storage system, the second inline compression producing a second set of compressed data and performed without storing the second set of data in the data object prior to compression;
store the first set of compressed data in the data object; and
store the second set of compressed data in the data object.

16. A computer program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a data storage system, cause the control circuitry to perform a method of managing storage space in a file system, the method comprising:
receiving multiple requests to write data to a data object stored in a data storage system, the data storage system including a set of processing units and memory;
performing a first inline compression of a first set of the data using software compression, the software compression carried out by executing a set of instructions stored in the memory by the set of processing units,

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Bassov Ivan
• Yim Wai C

Classifications

• G06F2212/621

  Coherency control relating to peripheral accessing, e.g. from DMA or I/O device · CPC title

• H03M7/60 Primary

  General implementation details not specific to a particular type of compression · CPC title

• G06F12/128

  adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel · CPC title

• H03M7/30 Primary

  Compression (speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N); Expansion; Suppression of unnecessary data, e.g. redundancy reduction · CPC title

• H03M7/3091

  Data deduplication · CPC title