Provenance-based replication in a storage system

Patent metadata

Field	Value
Publication number	US-11238063-B2
Application number	US-201916521728-A
Country	US
Kind code	B2
Filing date	Jul 25, 2019
Priority date	Jul 25, 2019
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

In one aspect, provenance-based replication includes assigning a GUID to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array. The trees are peers of each other with respect to at least one volume replicated between the arrays. For each volume in the first array that is replicated to a volume in the second array, an aspect includes assigning a volume pairing identifier common to both volumes. Upon determining data for a volume (V1) in the first array has been lost/corrupted, an aspect includes identifying the peer tree from the GUID and using the pairing ID of V1 to search the peer tree for a volume (V2) in the second array, retrieving data for V2, computing a delta between the data of V1 and the data of V2, and reconstructing the lost/corrupted data for V1 using the delta.

First claim

We claim:

1. A method for implementing provenance-based replication in a storage system, the method comprising:
assigning a globally unique identifier (GUID) to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array, wherein the first snap tree and the second snap tree are peers of each other with respect to at least one volume replicated between the first storage array and the second storage array;
for each volume (Vj) of a plurality of volumes in the first storage array that is replicated to a corresponding volume (Vi) of a plurality of volumes in the second storage array, assigning a volume pairing identifier (ID) common to both of the volume Vj and the volume Vi;
upon determining data for a volume (V 1 ) of the plurality of volumes in the first storage array has been lost or corrupted:
identifying the peer snap tree from the GUID of the peer snap tree and using the volume pairing ID of the volume V 1 to search the peer snap tree for a volume (V 2 ) of the plurality of volumes in the second storage array, the volume V 2 sharing the volume pairing ID of the volume V 1 ;
retrieving data for the volume V 2 and computing a delta between the data of the volume V 1 and the data of the volume V 2 ; and
reconstructing the lost or corrupted data for the volume V 1 using the delta.

2. The method of claim 1 , further comprising:
upon determining the volume V 1 and the volume V 2 are in sync and are read-only, setting an in-sync attribute to both the volume V 1 and the volume V 2 indicating the volume V 1 and the volume V 2 are currently in sync;
in response to performing a system recovery operation: and
accessing the in-sync attribute for the volume V 1 and the volume V 2 ;
upon determining the in-sync attribute is set for both the volume V 1 and the volume V 2 , bypassing a resynchronization operation on the volume V 1 and the volume V 2 ; and
upon determining the in-sync attribute is not set for at least one of the volume V 1 and the volume V 2 , performing the resynchronization operation on the volume V 1 and the volume V 2 .

3. The method of claim 2 , further comprising:
upon determining, during replication operations, at least one of the volume V 1 and the volume V 2 is out of sync, revoking the in-sync attribute for the volume V 1 and the volume V 2 .

4. The method of claim 2 , further comprising establishing the in-sync attribute for the volume V 1 and the volume V 2 during synchronous replication, the establishing comprising:
copying data from the volume V 1 to the volume V 2 ;
waiting for replication to finish copying the data from the volume V 1 to the volume V 2 ;
for writes received at the volume V 1 from a host after completion of the copying from the volume V 1 to the volume V 2 , replicating the writes from the volume V 1 to the volume V 2 ;
suspending, by the host, further writes to the volume V 1 ;
upon completion of replicating in-flight replication operations from the volume V 1 to the volume V 2 , taking a snapshot of the volume V 1 and taking a snapshot of the volume V 2 ;
providing provenance data to the snapshot of the volume V 1 and the snapshot of the volume V 2 , the provenance data including at least one of the GUID of the first snap tree and the second snap tree, a shared GUID pairing identifier of the first snap tree and the second snap tree, a current time and date, and the in-sync attribute; and
resuming writes from the host to the volume V 1 .

5. The method of claim 4 , wherein the shared GUID pairing identifier indicates that common ancestry data exists between the first snap tree and the second snap tree.

6. The method of claim 4 , further comprising replicating from an immutable volume (W 1 ) in the first storage array to the second storage array, the replicating comprising:
identifying, in the first storage array, the first snap tree corresponding to the volume W 1 ;
searching, in the second storage array, the second snap tree for volumes having common provenance data with provenance data of the volume W 1 ; and
upon determining no common provenance data is found in the second storage array, performing a full copy replication of the volume W 1 .

7. The method of claim 6 , further comprising:
upon determining common provenance is found in the second storage array, searching the first snap tree and the second snap tree for a set of volumes that are in sync with each other;
if no set of volumes are found to be in sync with each other, performing a full copy replication of volume W 1 ;
if the set of volumes are found to be in sync with each other:
taking a snapshot of a volume (W 2 ) off of a volume in the set residing in the second storage array;
establishing a replication relationship from the volume W 1 to the volume W 2 without copying data;
computing a difference set between the volume W 1 and a volume of the set residing in the first storage array;
sending the difference set to the second storage array; and
applying the difference set to the volume W 2 , the applying rendering the volume W 2 and the volume W 1 to be in sync.

8. A system for implementing provenance-based replication, the system comprising:
a memory comprising computer-executable instructions; and
a processor operable by a storage system, the processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:
assigning a globally unique identifier (GUID) to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array, wherein the first snap tree and the second snap tree are peers of each other with respect to at least one volume replicated between the first storage array and the second storage array;
for each volume (Vj) of a plurality of volumes in the first storage array that is replicated to a corresponding volume (Vi) of a plurality of volumes in the second storage array, assigning a volume pairing identifier (ID) common to both of the volume Vj and the volume Vi;
upon determining data for a volume (V 1 ) of the plurality of volumes in the first storage array has been lost or corrupted:
identifying the peer snap tree from the GUID of the peer snap tree and using the volume pairing ID of the volume V 1 to search the peer snap tree for a volume (V 2 ) of the plurality of volumes in the second storage array, the volume V 2 sharing the volume pairing ID of the volume V 1 ;
retrieving data for the volume V 2 and computing a delta between the data of the volume V 1 and the data of the volume V 2 ; and
reconstructing the lost or corrupted data for the volume V 1 using the delta.

9. The system of claim 8 , wherein the operations further comprise:
upon determining the volume V 1 and the volume V 2 are in sync and are read-only, setting an in-sync attribute to both the volume V 1 and the volume V 2 indicating the volume V 1 and the volume V 2 are currently in sync;
in response to performing a system recovery operation: and
accessing the in-sync attribute for the volume V 1 and the volume V 2 ;
upon determining the in-sync attribute is set for both the volume V 1 and the volume V 2 , bypassing a resynchronization operation on the volume V 1 and the volume V 2 ; and
upon determining the in-sync attribute is not set for at least one of the volume V 1 and the volume V 2 , performing the resynchronization operation on the volume V 1 and the volume V 2 .

10. The system of claim 9 , wherein the operations further comprise:
upon determining, during replication operations, at least one of the volume V 1 and the volume V

Assignees

• Emc Ip Holding Co Llc

Inventors

• Meiri David
• Kucherov Anton

Classifications

• G06F16/275 Primary

  Synchronous replication · CPC title

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title