Checking data integrity of data storage systems

Patent metadata

Field	Value
Publication number	US-10528416-B2
Application number	US-201715837098-A
Country	US
Kind code	B2
Filing date	Dec 11, 2017
Priority date	Dec 11, 2017
Publication date	Jan 7, 2020
Grant date	Jan 7, 2020

Abstract

A method and apparatus for validating operation of a data volume on a storage medium. A data integrity component is provided which writes data blocks to the volume in a sequence, each data block storing a sequence number and also write status information specifying the sequence numbers of those preceding data blocks in the stream which are still being written to the volume at the time the data block is generated. Data validation is performed by reading back the stored data blocks from the volume and checking that the sequence numbers stored in them match those that should be present based on the sequence numbers stored in the write status information of the last-written data block found on the volume.

First claim

What is claimed is:

1. A method for validating operation of a storage medium that is subdivided into a number of address blocks available for data storage in a computing environment, the method comprising:
writing a stream of data blocks in sequence to the storage medium, each data block being allocated a sequence number indicating its position in the stream, the sequence number being stored with the data block together with write status information specifying the sequence numbers of those preceding data blocks in the stream which are still being written to the storage medium at the time the data block is generated; and
assessing data integrity on the storage medium at any arbitrary point in time.

2. The method of claim 1 , further including:
generating the stream of data blocks in sequence to write to the storage medium;
allocating the address block to which a next data block in the stream is to be written by applying a sequencing function to the sequence number, wherein the sequencing function is periodic; and
performing a write process for each data block to its allocated address block.

3. The method of claim 1 , wherein assessing the data integrity further includes:
examining the storage medium to identify from the data blocks stored on the storage medium their sequence numbers;
identifying a last-written data block having a highest sequence number from among the identified data blocks;
extracting from the last-written data block its write status information;
determining from the set of sequence numbers and write status information that the identified data blocks are a match with those specified in the write status information; and
failing a test of the data upon failing to establish the match.

4. The method of claim 3 , further including determining the match by checking that the sequence numbers of the identified data blocks correspond to those that should be present on the storage medium based on the sequence numbers stored in the write status information of the last-written data block.

5. The method of claim 3 , wherein the write status information comprises a write status bitmap having a bit for each sequence number between a current sequence number and a lowest sequence number among one or more write processes that are incomplete, wherein each bit of the write status bitmap is set or unset depending on whether the write process for that sequence number is complete or incomplete, and wherein the data integrity is tested by constructing a validation bitmap having bits corresponding to the bits of the write status bitmap, wherein each bit of the validation bitmap is set or unset depending on whether a data block with that sequence number is found during the examining, and a bit-by-bit comparison of the write status bitmap and the validation bitmap.

6. The method of claim 5 , further including determining the data blocks are located in the storage medium for sequence numbers below the lowest sequence number among the one or more write processes that are incomplete stored in the write status information, at least within a range.

7. The method of claim 1 , further including:
permitting a maximum permitted number of write processes;
stalling a next write process upon reaching the maximum permitted number of write processes.

8. The method of claim 1 , wherein copying the first data further includes:
acquiring a point-in-time copy of the storage medium; and
assessing the data integrity using the point-in-time copy.

9. A system for validating operation of a storage medium that is subdivided into a number of address blocks available for data storage in a computing environment, comprising:
one or more computers with executable instructions that when executed cause the system to:
write a stream of data blocks in sequence to the storage medium, each data block being allocated a sequence number indicating its position in the stream, the sequence number being stored with the data block together with write status information specifying the sequence numbers of those preceding data blocks in the stream which are still being written to the storage medium at the time the data block is generated; and
assess data integrity on the storage medium at any arbitrary point in time.

10. The system of claim 9 , wherein the executable instructions further:
examine the storage medium to identify from the data blocks stored on the storage medium their sequence numbers;
identify a last-written data block having a highest sequence number from among the identified data blocks;
extract from the last-written data block its write status information;
determine from the set of sequence numbers and write status information that the identified data blocks are a match with those specified in the write status information; and
fail a test of the data upon failing to establish the match.

11. The system of claim 10 , wherein the executable instructions further determine the match by checking that the sequence numbers of the identified data blocks correspond to those that should be present on the storage medium based on the sequence numbers stored in the write status information of the last-written data block.

12. The system of claim 10 , wherein the write status information comprises a write status bitmap having a bit for each sequence number between a current sequence number and a lowest sequence number among one or more write processes that are incomplete, wherein each bit of the write status bitmap is set or unset depending on whether the write process for that sequence number is complete or incomplete, and wherein the data integrity is tested by constructing a validation bitmap having bits corresponding to the bits of the write status bitmap, wherein each bit of the validation bitmap is set or unset depending on whether a data block with that sequence number is found during the examining, and a bit-by-bit comparison of the write status bitmap and the validation bitmap.

13. The system of claim 12 , wherein the executable instructions further determine the data blocks are located in the storage medium for sequence numbers below the lowest sequence number among the one or more write processes that are incomplete stored in the write status information, at least within a range.

14. The system of claim 9 , wherein the executable instructions further:
permit a maximum permitted number of write processes;
stall a next write process upon reaching the maximum permitted number of write processes.

15. The system of claim 9 , wherein the executable instructions further:
acquire a point-in-time copy of the storage medium; and
assess the data integrity using the point-in-time copy.

16. A computer program product for validating operation of a storage medium that is subdivided into a number of address blocks available for data storage by a processor, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein executed by the processor, the computer-readable program code portions comprising:
an executable portion that writes a stream of data blocks in sequence to the storage medium, each data block being allocated a sequence number indicating its position in the stream, the sequence number being stored with the data block together with write status information specifying the sequence numbers of those preceding data blocks in the stream which are still being written to the storage medium at the time the data block is generated; and
an executable portion that assesses data integrity on the storage medium at any arbitrary point

Assignees

• IBM

Inventors

• Wilkinson John
• Sipka Andrea

Classifications

• G06F11/2221 Primary

  to test input/output devices or peripheral units · CPC title

• G06F11/0751

  Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06) · CPC title

• G06F11/079 Primary

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title

• G06F11/0727

  in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title