Eliminating duplicate data by sharing file system extents

Patent metadata

Field	Value
Publication number	US-9483487-B2
Application number	US-201414456914-A
Country	US
Kind code	B2
Filing date	Aug 11, 2014
Priority date	Nov 30, 2009
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

A hardware and/or software facility to enable emulated storage devices to share data stored on physical storage resources of a storage system. The facility may be implemented on a virtual tape library (VTL) system configured to back up data sets that have a high level of redundancy on multiple virtual tapes. The facility organizes all or a portion of the physical storage resources according to a common store data layout. By enabling emulated storage devices to share data stored on physical storage resources, the facility enables deduplication across the emulated storage devices irrespective of the emulated storage device to which the data is or was originally written, thereby eliminating duplicate data on the physical storage resources and improving the storage consumption of the emulated storage devices on the physical storage resources.

First claim

We claim:

1. A physical storage device comprising:
a first data set written to a first emulated storage device, wherein the first data set includes a first section of data; and
a second data set written to a second emulated storage device, wherein the second data set includes the first section of data, and wherein the first section of data is written to a storage location on the physical storage device that is shared by the first and second emulated storage devices,
wherein the physical storage device is organized according to a common store data layout, and wherein the storage location corresponds to a portion of a common store extent, said common store extend including a plurality of common store units.

2. The physical storage device of claim 1 wherein the common store extent is referenced by an offset in a common store unit, an uncompressed length of the common store extent, and a number of common store units included in the common store extent.

3. The physical storage device of claim 1 wherein each of the plurality of common store units includes a fixed number of contiguous data blocks.

4. The physical storage device of claim 1 wherein each of the plurality of common store units includes a variable number of contiguous data blocks.

5. The physical storage device of claim 1 wherein each of the plurality of common store units includes a header, one or more record compression units of compressed data, and boundary information mapping the beginning of each record compression unit to an offset in compressed storage of the physical storage device.

6. The physical storage device of claim 1 wherein the emulated storage devices are virtual tapes of a virtual tape library.

7. The physical storage device of claim 1 wherein a deduplication technique is applied to the physical storage device to eliminate duplicate data stored thereon irrespective of the emulated storage device to which the duplicate data was originally written.

8. A physical storage device comprising:
a first data set written to a first emulated storage device, wherein the first data set includes a first section of data; and
a second data set written to a second emulated storage device, wherein the second data set includes the first section of data, and wherein the first section of data is written to a storage location on the physical storage device that is shared by the first and second emulated storage devices,
wherein the physical storage device is organized according to a common store data layout, and wherein the storage location corresponds to a portion of a common store extent that is referenced in whole by the first emulated storage device, and referenced in part by the second emulated storage device.

9. The physical storage device of claim 8 , wherein the emulated storage devices are virtual tapes of a virtual tape library.

10. The physical storage device of claim 8 , wherein a deduplication technique is applied to the physical storage device to eliminate duplicate data stored thereon irrespective of the emulated storage device to which the duplicate data was originally written.

11. One or more computer-readable memories storing computer-executable instructions for implementing a method for deduplication at a storage server, comprising:
instructions for storing a first data set at a first emulated storage device, wherein the first data set includes a first section of data; and
instructions for storing a second data set at a second emulated storage device, wherein the second data set includes the first section of data, and wherein the first section of data is stored at a storage location on the physical storage device that is shared by the first and second emulated storage devices,
wherein the physical storage device is organized according to a common store data layout, and wherein the storage location corresponds to a portion of a common store extent, the common store extent including a plurality of common store units.

12. The one or more computer-readable memories of claim 11 , wherein each of the plurality of common store units includes a header, one or more record compression units of compressed data, and boundary information mapping the beginning of each record compression unit to an offset in compressed storage of the physical storage device.

13. The one or more computer-readable memories of claim 11 , wherein the common store extent is referenced by an offset in a common store unit, an uncompressed length of the common store extent, and a number of common store units included in the common store extent.

14. The one or more computer-readable memories of claim 11 , further comprising:
instructions for applying a deduplication technique to the physical storage device to eliminate duplicate data stored thereon irrespective of the emulated storage device to which the duplicate data was originally written.

15. One or more computer-readable memories storing computer-executable instructions for implementing a method for deduplication at a storage server, comprising:
instructions for storing a first data set at a first emulated storage device, wherein the first data set includes a first section of data; and
instructions for storing a second data set at a second emulated storage device, wherein the second data set includes the first section of data, and wherein the first section of data is stored at a storage location on the physical storage device that is shared by the first and second emulated storage devices,
wherein the physical storage device is organized according to a common store data layout, and wherein the storage location corresponds to a portion of a common store extent that is referenced in whole by the first emulated storage device, and referenced in part by the second emulated storage device.

Assignees

• Netapp Inc

Inventors

• Gupta Vivek
• Pyati Ameet
• Singhal Satish
• Saxena Pawan

Classifications

• G06F3/0641 Primary

  De-duplication techniques · CPC title

• G06F2003/0694

  Physics · mapped topic

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F17/30156 Primary

  Physics · mapped topic

• G06F3/0686

  Libraries, e.g. tape libraries, jukebox · CPC title