Compressed data verification

Patent metadata

Field	Value
Publication number	US-11243890-B2
Application number	US-202016742201-A
Country	US
Kind code	B2
Filing date	Jan 14, 2020
Priority date	Jan 14, 2020
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

Embodiments of the present disclosure relate to verifying compressed data. Compressed data files can be read from a global cache for a storage device into a local buffer. A data verification level of a plurality of data verification levels can be selected to perform on the compressed data files. An amount of data blocks of each data file can be decompressed based on the determined data verification level. An integrity of the compressed data files verified using the decompressed data blocks.

First claim

What is claimed is:

1. An apparatus configured to at least one processor configured to:
read compressed data files from a global cache for a storage device into a local buffer;
select a data verification level of a plurality of data verification levels to perform on the compressed data files, wherein selecting the data verification level includes determining telemetry information of the storage device;
decompress an amount of data blocks of each data file based on the determined data verification level; and
verify an integrity of the compressed data files using the decompressed data blocks.

2. The apparatus of claim 1 , wherein the plurality of data verification levels includes at least a first compression verification level and a second verification level, wherein each verification level requires distinct computing resources and a distinct verification response time.

3. The apparatus of claim 1 , wherein the selected data verification level is based on the determined telemetry information.

4. The apparatus of claim 1 further configured to decompress a single data block of each read compressed data file in response to selecting a data verification level having a verification response time less than a threshold.

5. The apparatus of claim 4 further configured to compare each single decompressed data block of each compressed data file to at least one data block of a corresponding uncompressed data file stored in global memory.

6. The apparatus of claim 5 , wherein:
the comparison includes determining a similarity between information contained a cyclic redundancy check (CRC) field and a data integrity field (DIF) of the single data block and the at least one data block of the corresponding uncompressed data file;
a similarity finding indicates that the read compressed data files from the global cache are verified; and
a lack of a similarity finding indicates that the read compressed data files from the global cache are unverified.

7. The apparatus of claim 1 further configured to decompress two or more data blocks of each read compressed data file in response to selecting a data verification level having a verification response time greater than a threshold.

8. The apparatus of claim 7 further configured to:
compare each of the two or more decompressed data block of each compressed data file to a plurality of data blocks of a corresponding uncompressed data file stored in global memory; and
wherein:
the comparison includes determining a similarity between information contained a cyclic redundancy check (CRC) field and a data integrity field (DIF) of a single data block and the plurality of data blocks of the corresponding uncompressed data file,
a similarity finding indicates that the read compressed data files from the global cache are verified, and
lack of a similarity finding indicates that the read compressed data files from the global cache are unverified.

9. The apparatus of claim 1 further configured to write all verified compressed data files to one or more disks of the storage device.

10. A method comprising:
reading compressed data files from a global cache for a storage device into a local buffer;
selecting a data verification level of a plurality of data verification levels to perform on the compressed data files, wherein selecting the data verification level includes determining telemetry information of the storage device;
decompressing an amount of data blocks of each data file based on the determined data verification level; and
verifying an integrity of the compressed data files using the decompressed data blocks.

11. The method of claim 10 , wherein the plurality of data verification levels includes at least a first compression verification level and a second verification level, wherein each verification level requires distinct computing resources and a distinct verification response time.

12. The method of claim 10 , wherein the selected data verification level is based on the determined telemetry information.

13. The method of claim 10 further comprising decompressing a single data block of each read compressed data file in response to selecting a data verification level having a verification response time less than a threshold.

14. The method of claim 13 further comprising comparing each single decompressed data block of each compressed data file to at least one data block of a corresponding uncompressed data file stored in global memory.

15. The method of claim 14 , wherein:
the comparison includes determining a similarity between information contained a cyclic redundancy check (CRC) field and a data integrity field (DIF) of the single data block and the at least one data block of the corresponding uncompressed data file;
a similarity finding indicates that the read compressed data files from the global cache are verified; and
lack of a similarity finding indicates that the read compressed data files from the global cache are unverified.

16. The method of claim 10 further comprising decompressing two or more data blocks of each read compressed data file in response to selecting a data verification level having a verification response time greater than a threshold.

17. The method of claim 16 further comprising:
comparing each of the two or more decompressed data block of each compressed data file to a plurality of data blocks of a corresponding uncompressed data file stored in global memory; and
wherein:
the comparison includes determining a similarity between information contained a cyclic redundancy check (CRC) field and a data integrity field (DIF) of a single data block and the plurality of data blocks of the corresponding uncompressed data file,
a similarity finding indicates that the read compressed data files from the global cache are verified, and
lack of a similarity finding indicates that the read compressed data files from the global cache are unverified.

18. The method of claim 10 further comprising writing all verified compressed data files to one or more disks of the storage device.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Wu Peng
• Yu Rong
• Gong Tao

Classifications

• G06F11/1004

  to protect a block of data words, e.g. CRC or checksum (G06F11/1076 takes precedence; security arrangements for protecting computers or computer systems against unauthorized activity G06F21/00) · CPC title

• G06F11/3034 Primary

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/3058

  Monitoring arrangements for monitoring environmental properties or parameters of the computing system or of the computing system component, e.g. monitoring of power, currents, temperature, humidity, position, vibrations (thermal management in cooling arrangements of a computing system G06F1/206) · CPC title

• G06F12/0897 Primary

  with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811) · CPC title

• G06F2212/401

  Compressed data · CPC title