Deduplicating container files

Patent metadata

Field	Value
Publication number	US-9569455-B1
Application number	US-201313930010-A
Country	US
Kind code	B1
Filing date	Jun 28, 2013
Priority date	Jun 28, 2013
Publication date	Feb 14, 2017
Grant date	Feb 14, 2017

Abstract

A technique forms contents of a deduplication container from a standalone container file system stored on the same data storage array on which the deduplication container is stored. Such a standalone container includes a container file which is formed by standalone blocks that are not shared with any other file within the standalone container outside of snapshots of that file. The standalone blocks store host data and which are hierarchically related to each other. Upon receiving a deduplication command, the storage processor carries out a set of deduplication operations which include creating and storing the deduplication container in the data storage array. The deduplication container includes container files which are formed by deduplication blocks storing the host data and which are hierarchically related to each other.

First claim

What is claimed is:

1. A method of performing deduplication on a data storage array that stores host data, the method comprising:
initially storing a standalone container on the data storage array, the standalone container including container files which are formed by standalone blocks storing the host data and which are hierarchically related to each other;
receiving a deduplication command which identifies the standalone container; and
in response to the deduplication command, performing a set of deduplication operations to create and store a deduplication container on the data storage array, the deduplication container including container files which are formed by deduplication blocks storing the host data and which are hierarchically related to each other,
wherein each of the standalone container and the deduplication container is a lower deck file system in the data storage array, each container file of the standalone container and the deduplication container representing an upper deck file system for a host file system accessible to a respective host computer of a set of host computers;
wherein the deduplication command which identifies the standalone container is received from a host computer of the set of host computers.

2. A method as in claim 1 ,
wherein performing the set of deduplication operations includes:
assigning a virtual block map (VBM) pointer to each standalone block that is not marked as a duplicate to form a deduplication block,
for each standalone block marked as a duplicate, i) assigning a VBM pointer to another VBM pointer that points to a standalone block that is not marked as a duplicate and contains the same host data as that standalone block, and ii) freeing that standalone block.

3. A method as in claim 2 ,
wherein performing the set of deduplication operations further includes:
establishing a replication session between the standalone container and the deduplication container, and
after establishing the replication session, replicating the container file of the standalone container in the deduplication container; and
wherein freeing each standalone block includes:
removing the standalone container from the data storage array.

4. A method as in claim 3 ,
wherein the standalone container includes a particular container file and a set of snapshots of the particular container file, each of the set of snapshots being identified with an elapsed time since creation;
wherein establishing the replication session includes:
locating an oldest snapshot of the set of snapshots, the oldest snapshot being identified with the longest elapsed time since creation, and
identifying subsequent snapshots in descending order of elapsed time since creation.

5. A method as in claim 4 ,
wherein replicating the full contents of the standalone container in the deduplication container includes:
replicating the oldest snapshot in the deduplication container;
for each subsequent snapshot, i) forming a difference between that subsequent snapshot and the previous snapshot, ii) replicating the difference in the deduplication container, and iii) adding the replicated difference to the replicated previous snapshot to form a replicated subsequent snapshot; and
wherein removing the standalone container from the data storage array includes:
deleting the set of snapshots from the standalone container.

6. A method as in claim 4 ,
wherein the standalone container includes an external replication session having a replication configuration between the standalone container and a replicated standalone container on another data storage array;
wherein the method further comprises:
prior to removing the standalone container from the data storage array, i) pausing the external replication session and ii) copying the replication configuration of the external replication session to the deduplication container;
deleting the replication configuration from the standalone container; and
establishing another external replication session between the deduplication container and the replicated standalone container.

7. A method as in claim 4 ,
wherein the standalone container includes an external replication session having a replication configuration between the standalone container and a replicated standalone container on another data storage array
wherein the standalone container acts as a source file system and the other data storage array as a target file system, the standalone container having a replication configuration between the standalone container and the replicated standalone container on another data storage array;
wherein establishing the replication session includes:
setting up a symmetric replication session involving the external replication session and the deduplication container;
wherein the method further comprises:
prior to removing the standalone container from the data storage array, establishing a failover condition in the standalone container, the failover condition causing the deduplication container to act as the source file system; and
deleting the replication configuration from the standalone container.

8. A method as in claim 1 ,
wherein the data storage array includes an upper deck file system accessible to a host and a lower deck file system including the standalone container and the deduplication container, the upper deck file system being stored in the form of a container file of the standalone container;
wherein receiving the deduplication command which identifies the standalone container includes:
obtaining a message from the host indicating that the container file of the standalone container is to be converted into a container file in the deduplication container.

9. A method as in claim 8 ,
wherein the lower deck file system is provisioned blocks of data from a storage pool, the storage pool containing all blocks of storage of the data storage array, a single deduplication container being stored per storage pool; and
wherein performing the set of deduplication operations to create and store the deduplication container on the data storage array includes:
prior to creating the deduplication container, verifying whether there is another deduplication container stored in the lower deck file system.

10. A data storage apparatus constructed and arranged to store host data, the data storage system comprising:
a set of storage devices; and
a storage processor, the storage processor including:
memory; and
a set of processors coupled to the memory to form controlling circuitry, the controlling circuitry constructed and arranged to:
initially store a standalone container on the data storage array, the standalone container including container files which are formed by standalone blocks storing the host data and which are hierarchically related to each other;
receive a deduplication command which identifies the standalone container; and
in response to the deduplication command, perform a set of deduplication operations to create and store a deduplication container on the data storage array, the deduplication container including container files which are formed by deduplication blocks storing the host data and which are hierarchically related to each other,
wherein each of the standalone container and the deduplication container is a lower deck file system in the data storage array, each container file of the standalone container and the deduplication container representing an upper deck file system for a host file system accessible to a respective host computer of a set of host computers;
wherein the deduplication command which identifies the standalone container is received from a host computer of the set of host computers.

11.

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Bono Jean-Pierre
• Davenport William C
• De Forest Miles A
• Armangau Philippe
• Forrester Walter C
• Owen Karl M
• Tummala Himabindu

Classifications

• G06F3/0689

  Disk arrays, e.g. RAID, JBOD · CPC title

• G06F16/1748 Primary

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0641

  De-duplication techniques · CPC title

• G06F3/0611

  in relation to response time · CPC title