Data storage system with fixity end-to-end data protection during data archiving

Patent metadata

Field	Value
Publication number	US-2016378772-A1
Application number	US-201514748398-A
Country	US
Kind code	A1
Filing date	Jun 24, 2015
Priority date	Jun 24, 2015
Publication date	Dec 29, 2016
Grant date	—

Abstract

A data storage system is provided that is configured to achieve end-to-end data protection. The system includes a server running a storage archive manager with a first fixity support module, and the system includes a tape drive running a second fixity support module. During operations, the storage archive manager reads a data file with associated fixity information from a disk storage device, and the storage archive manager builds a set of blocks from the data file for writing to the tape drive. The first fixity support module checks fixity of the set of blocks by processing the fixity information associated with the data file. The processing of the fixity information includes determining whether data in the set of blocks is out of order relative to data in the data file or whether data in the set of blocks is corrupt relative to the data in the data file.

First claim

We claim:

1 . A data storage system with end-to-end data protection, comprising:
a server with a processor running a storage archive manager with a first fixity support module; and a tape drive for writing data to a magnetic storage tape, wherein the tape drive includes a processor running a second fixity support module, wherein the storage archive manager reads a data file with associated fixity information from a disk storage device, wherein the storage archive manager builds a set of blocks from the data file for transfer to the tape drive; and wherein the first fixity support module checks fixity the set of blocks by processing the fixity information associated with the data file.

2 . The data storage system of claim 1 , wherein the processing of the fixity information associated with the data file by the first fixity support module includes determining whether data in the set of blocks is out of order relative to data in the data file or whether data in the set of blocks is corrupt relative to the data in the data file.

3 . The data storage system of claim 1 , wherein the first fixity support module generates an accumulated checksum or hash value based on checksum or hash values generated for each of the blocks in the set of blocks and wherein the processing comprises comparing the accumulated checksum or hash value with the fixity information associated with the data file.

4 . The data storage system of claim 3 , wherein the fixity information associated with the data file and the checksum or hash values generated for each of the blocks both comprise an SHA-1 or SHA-256 checksum.

5 . The data storage system of claim 1 , wherein the first fixity support module generates a first accumulated checksum or hash value of check values associated with each of the blocks in the set of blocks, wherein the second fixity support module generates a second accumulated checksum or hash value of check values associated with the blocks after transfer to the tape drive, and wherein the first fixity support module compares the first and second accumulated checksums or hash values of check values to determine whether after the transfer to the tape drive the blocks include corrupt data.

6 . The data storage system of claim 5 , wherein the check values each comprises a cycle redundancy check (CRC) value.

7 . The data storage system of claim 5 , wherein the determination of whether the blocks include corrupt data is performed free of a read back of data in the blocks written to the magnetic storage tape to the storage archive manager.

8 . A method of checking fixity during data transfers, comprising:
reading a data file from a first data storage device; building two or more records from the data file; determining fixity information for each of the records; accumulating the determined fixity information into an accumulated fixity value for the records; and comparing the accumulated fixity value with fixity information in metadata associated with the data file received from the first data storage device.

9 . The method of claim 8 , wherein the first data storage device is a disk storage device and the reading, the building, the determining, the accumulating, and the comparing steps are performed by a metadata server running a storage archive manager and a fixity support module.

10 . The method of claim 9 , wherein the comparing is used by the fixity support module to determine whether data in the built records is corrupt or out of order relative to data in data in the data file in the first data storage device.

11 . The method of claim 8 , wherein the determined fixity information and the fixity information in the metadata are of matching formats and each comprises an SHA-1 or SHA-256 checksum.

12 . The method of claim 8 , wherein the building step comprises determining a CRC for each of the records and wherein the method further comprises:
accumulating checksum or hash values of all the CRCs into an accumulated CRC checksum or hash value; transferring the records to a second data storage device; at the second data storage device, determining a CRC for each of the transferred records; at the second data storage device, accumulating checksum or hash values of all the CRCs for the transferred records into an accumulated CRC checksum or hash value for the transferred records; comparing the accumulated CRC checksums or hash values to check fixity.

13 . The method of claim 12 , wherein the accumulating, the transferring, and the comparing steps are performed by a metadata server positioned between the first and second data storage devices, wherein the first data storage device comprises a disk storage devices, and wherein the second data storage device comprises a tape drive.

14 . A system for providing end-to-end data protection during archiving, comprising:
a disk storage device storing a data file and fixity information for the data file; a metadata server providing a first fixity support module; and a tape drive providing a second fixity support module, wherein the metadata server reads the data file, builds a set of blocks from the data file, and generates fixity information for the blocks with the first fixity support module; wherein the tape drive receives the blocks for writing to tape media and, in response, generates fixity information for the received blocks with the second fixity support module; wherein the first fixity support module checks fixity between the disk storage device and the metadata server by comparing the fixity information for the data file with the fixity information generated at the metadata server for the blocks; and wherein the first fixity support module checks fixity between the metadata server and the tape drive by comparing the fixity information generated at the metadata server for the blocks with the fixity information generated at the tape drive for the received blocks.

15 . The system of claim 14 , wherein the fixity information for the data file comprises a checksum or hash value and the fixity information generated at the metadata server for the blocks comprises an accumulated checksum or hash value calculated by accumulating checksums or hash values generated by the first fixity support module for each of the blocks.

16 . The system of claim 15 , wherein the checksums or hash values comprise SHA-1 or SHA-256 checksums or hash values.

17 . The system of claim 15 , wherein the first fixity support module determines data is out of order, data is corrupt, or fixity is maintained based on the comparing of the fixity information for the data file with the fixity information generated at the metadata server for the blocks.

18 . The system of claim 14 , wherein the fixity information generated at the metadata server for the blocks comprises an accumulated checksum or hash value calculated by accumulating check values generated for each of the blocks and wherein the fixity information generated at the tape drive for the received blocks comprises an accumulated checksum or hash value calculated by accumulating check values generated at the tape drive for each of the received blocks.

19 . The system of claim 18 , wherein the check values each comprise a CRC.

20 . The system of claim 18 , wherein the first fixity support module determines data is corrupt or fixity is maintained based on the comparing of the fixity information generated at the metadata server for the blocks with the fixity information generated at the tape drive for the received blocks.

Assignees

• Oracle Int Corp

Inventors

• Hostetter David G
• Major David Allen

Classifications

• G06F16/113 Primary

  Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446) · CPC title

• G06F16/1794

  Details of file format conversion · CPC title

• G06F17/30073 Primary

  Physics · mapped topic

• G06F17/30371

  Physics · mapped topic

• G06F17/30109

  Physics · mapped topic