Tape drive library integrated memory deduplication

Patent metadata

Field	Value
Publication number	US-10656860-B2
Application number	US-201916271271-A
Country	US
Kind code	B2
Filing date	Feb 8, 2019
Priority date	Sep 12, 2017
Publication date	May 19, 2020
Grant date	May 19, 2020

Abstract

A method and system for improving tape drive memory storage is provided. The method includes receiving, by a storage tape drive hardware device, a data stream. Duplicate data chunks of the data stream are identified and deleted such that a group of data chunks remain in a non-volatile memory device (NVS 2 ) of the storage tape drive. The group of data chunks is written to a data storage tape cartridge. Pointers are generated and stored within the data storage tape cartridge. The pointers are associated with a location within an NVS 1 for storing de-duplication hashes identifying each data chunk of the group of data chunks stored within data storage tape cartridge. The de-duplication hashes are written from the NVS 1 to a manager non-volatile memory device (MNVS) of a storage tape drive hardware library system. The MNVS is updated.

First claim

What is claimed is:

1. A tape drive memory storage improvement method comprising:
receiving, by a processor of a storage tape drive hardware device of a storage tape drive hardware library system comprising a plurality of storage tape drive hardware devices, a data stream for storage, wherein said storage tape drive hardware device internally comprises a deduplication software engine, a non-volatile memory device (NVS 1 ), a non-volatile memory device (NVS 2 ), and a first data storage tape cartridge;
identifying, by said processor within said NVS 2 , duplicate data chunks of a plurality of adjacent variable length data chunks of said data stream, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
deleting, by said processor from said NVS 2 , said duplicate data chunks such that said first group of data chunks remain within said NVS 2 ;
writing, by said processor to a first data storage tape cartridge of said storage tape drive hardware device, said first group of data chunks;
generating, by said processor for storage with said first data storage tape cartridge, pointers associated with a location within said NVS 1 storing de-duplication hashes identifying each data chunk of said first group of data chunks stored within said first data storage tape cartridge;
writing, by said processor from said NVS 1 to a manager non-volatile memory device (MNVS) of said storage tape drive hardware library system, said de-duplication hashes; and
updating, by said processor, said MNVS by combining said de-duplication hashes with a plurality of additional de-duplication hashes identifying a plurality of commonly used de-duplicated data chunks stored within said plurality of storage tape drive hardware devices.

2. The method of claim 1 , wherein said first data storage tape cartridge is removed from said storage tape drive hardware device, wherein a second data storage tape cartridge is placed within a second storage tape drive hardware device of said plurality of storage tape drive hardware devices, and wherein said method further comprises:
writing, from said MNVS to an additional NVS 1 of said second storage tape drive hardware device, said de-duplication hashes combined with said plurality of additional de-duplication hashes;
receiving an additional data stream for storage;
dividing said additional data stream into a second plurality of adjacent variable length data chunks;
storing, within an additional NVS 2 of said second storage tape drive hardware device, said second plurality of adjacent variable length data chunks;
de-duplicating, based on said de-duplication hashes combined with said plurality of additional de-duplication hashes, said second plurality of adjacent variable length data chunks such that a second group of de-duplicated data chunks remain within said additional NVS 2 ;
writing, by said processor to said second data storage tape cartridge of said second storage tape drive hardware device, said second group of de-duplicated data chunks;
generating, by said processor, additional pointers associated with a location within said additional NVS 1 storing said de-duplication hashes combined with said plurality of additional de-duplication hashes; and
storing, by said processor, said pointers within said second data storage tape cartridge.

3. The method of claim 2 , wherein said storage tape drive hardware library system further comprises a communication component, and wherein said method further comprises:
synchronizing with all de-duplication hashes within said MNVS, via said communication component, all additional de-duplication hashes associated with all additional de-duplicated data chunks processed by said plurality of storage tape drive hardware devices and said storage tape drive hardware library system such that said MNVS is updated with currently updated de-duplication hashes associated with currently updated de-duplicated data chunks.

4. The method of claim 2 , further comprising:
detecting corrupted data chunks stored within a storage tape drive hardware device of said plurality of storage tape drive hardware devices;
retrieving, from said MNVS, pointers associated with functional data chunks of at least one additional NVS 2 of said plurality of storage tape drive hardware devices comprising data storage tape cartridges including said functional data chunks;
replacing said corrupted data chunks with said functional data chunks.

5. The method of claim 1 , further comprising:
providing an Ethernet connection between said MNVS and said plurality of storage tape drive hardware devices.

6. The method of claim 1 , wherein said MNVS comprises a memory device selected from the group consisting of an integrated circuit based memory device and a removable flash memory device.

7. The method of claim 1 , wherein said MNVS comprises a hash database.

8. The method of claim 1 , further comprising:
encrypting, by said processor, all hashes within said MNVS.

9. The method of claim 1 , further comprising:
providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable code in the storage tape drive hardware device, said code being executed by the processor to implement: said receiving, said identifying, said deleting, said writing said first group of data chunks, said generating said pointers, said writing said de-duplication hashes, and said updating.

10. A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, said computer readable program code comprising an algorithm that when executed by a processor of a storage tape drive hardware device implements a tape drive memory storage improvement method, said method comprising:
receiving, by said processor, a data stream for storage, wherein said storage tape drive hardware device is comprised by a storage tape drive hardware library system comprising a plurality of storage tape drive hardware devices, and wherein said storage tape drive hardware device internally comprises a deduplication software engine, a non-volatile memory device (NVS 1 ), a non-volatile memory device (NVS 2 ), and a first data storage tape cartridge;
identifying, by said processor within said NVS 2 , duplicate data chunks of a plurality of adjacent variable length data chunks of said data stream, wherein said duplicate data chunks comprise duplicated data with respect to a first group of data chunks of said plurality of adjacent variable length data chunks;
deleting, by said processor from said NVS 2 , said duplicate data chunks such that said first group of data chunks remain within said NVS 2 ;
writing, by said processor to a first data storage tape cartridge of said storage tape drive hardware device, said first group of data chunks;
generating, by said processor for storage with said first data storage tape cartridge, pointers associated with a location within said NVS 1 storing de-duplication hashes identifying each data chunk of said first group of data chunks stored within said first data storage tape cartridge;
writing, by said processor from said NVS 1 to a manager non-volatile memory device (MNVS) of said storage tape drive hardware library system, said de-duplication hashes; and
updating, by said processor, said MNVS by combining said de-duplication hashes with a plurality of additional de-duplication hashes identifying a plurality of commonly used de-duplicated data chunks stored within said plurality of storage tape drive hardware devices.

11. The computer program product of claim 10 , w

Assignees

• IBM

Inventors

• Asmussen Ole
• Beiderbeck Robert
• Rueger Erik
• Schäfer Markus

Classifications

• G06F3/0686

  Libraries, e.g. tape libraries, jukebox · CPC title

• G06F2201/82

  Solving problems relating to consistency · CPC title

• G06F3/0641 Primary

  De-duplication techniques · CPC title

• G06F2201/805

  Real-time · CPC title

• G06F3/0608

  Saving storage space on storage systems · CPC title