Snapshot-based remote replication

Patent metadata

Field	Value
Publication number	US-11893260-B2
Application number	US-202117238103-A
Country	US
Kind code	B2
Filing date	Apr 22, 2021
Priority date	Apr 22, 2021
Publication date	Feb 6, 2024
Grant date	Feb 6, 2024

Abstract

Snapshots may be used to implement remote replication, for example, asynchronously, between a first storage system, A1, and a second storage system, A2. A1 may take a first snapshot, SS2 1 , of a logical storage unit, R1. After the snapshot SS2 1 it taken, any dependent write operations associated with SS2 1 may be reconciled, and differences between SS2 1 and the last snapshot for R1, SS1 1 , may be determined and recorded as a difference set. One or more replication instructions for R1 that include the write operations (or data and metadata corresponding thereto) of the difference set may be sent from A1 to A2. A2 may apply the differences to R2, and then take (activate) a snapshot of R2, SS2 2 , which is a replica of SS2 1 . After A2 activates SS2 2 , A2 may send an acknowledgement to A1 indicating that SS2 2 has been activated, and A2 may take a next snapshot of R1.

First claim

What is claimed is:

1. A method of replicating data from a first storage system to a second storage system, comprising:
receiving, at the first storage system from a host, a write request that writes first content on a first logical storage unit;
responsive to the first storage system receiving the write request from the host, the first storage system partitioning the write request into multiple dependent writes, wherein each of the multiple dependent writes is dependent on all other ones of the multiple dependent writes such that the multiple dependent writes are all included in a same snapshot of the first logical storage unit;
taking, at a first point in time, a first snapshot of the first logical storage unit on the first storage system, the first logical storage unit including a plurality of logical storage elements;
prior to taking the first snapshot of the first logical storage unit at the first point in time, performing a first of the multiple dependent writes to the first logical storage unit;
determining that one or more dependent writes of the multiple dependent writes are not performed on the first logical storage unit prior to said taking the first snapshot of the first logical storage unit at the first point in time;
reconciling the one or more dependent writes of the first snapshot prior to determining a first difference set of differences between the first snapshot of the first logical storage unit and a second snapshot of the first logical storage unit, wherein the second snapshot of the first logical storage unit is a most recent previous snapshot of the first logical storage unit taken at a point in time prior to the first point in time, wherein said reconciling includes:
delaying said determining the first difference set until the one or more dependent writes are performed on the first logical storage unit and until the first snapshot is updated to include the one or more dependent writes;

subsequent to completing said reconciling, performing said determining the first difference set of differences between the first snapshot of the first logical storage unit and the second snapshot of the first logical storage unit, wherein said determining the first difference set further includes:
for at least a first logical storage element of the first logical storage unit, determining if first snapshot metadata of the first snapshot references first data that also is referenced for the at least first logical storage element by second snapshot metadata of the second snapshot denoting the most recent previous snapshot of the first logical storage unit;

sending from the first storage system to the second storage system a replication instruction, wherein, if the first data is not also referenced by the second snapshot metadata, the replication instruction specifies the first logical storage element and the first data; and
applying the replication instruction on the second storage system.

2. The method of claim 1 , wherein said determining if the first snapshot metadata references the first data that also is referenced by the second snapshot metadata includes determining if the first snapshot metadata references a first physical storage location that also is referenced for the at least first logical storage element by the second snapshot metadata.

3. The method of claim 1 , wherein a second logical storage unit on the second storage system is a remote replica of the first logical storage unit, and wherein said applying the replication instruction on the second storage system includes:
accessing metadata of the second logical storage unit that specifies a physical storage location on the second storage system for current data of the second logical storage unit; and
storing the first data at the physical storage location.

4. The method of claim 1 , wherein a second logical storage unit on the second storage system is a remote replica of the first logical storage unit, wherein said replication instruction includes writing the first data to the first logical storage element, and wherein said applying the replication instruction on the second storage system includes writing the first data to the first logical storage element of the second logical storage unit, and wherein the method further comprises:
after applying the replication instruction on the second storage system, taking a snapshot of the second logical storage unit on the second storage system.

5. The method of claim 1 , further comprising:
for each of one or more logical storage units on the first storage system that are members of a snapshot group, including the first logical storage unit, taking a respective snapshot of the logical storage unit, including the first snapshot of the first logical storage unit; and
for each of one or more logical storage units, determining if, for any of the logical storage elements of the logical storage unit, snapshot metadata of the respective snapshot data of the logical storage unit references data that also is referenced for the logical storage element by snapshot metadata of a most recent previous snapshot of the respective logical storage unit of the logical storage element,
wherein, for any logical storage element of the one or more logical storage units for which the data is not also referenced by the snapshot metadata of a most recent previous snapshot of the respective logical storage unit of the logical storage element, the replication instruction specifies the first logical storage element and the data.

6. The method of claim 1 , further comprising:
the first storage system receiving an acknowledgement from the second storage system that the application of the replication instruction has been performed; and
in response to receiving the acknowledgement, taking another second snapshot of the first logical storage unit on the first storage system.

7. The method of claim 1 , wherein the first logical storage unit is one of one or more logical storage units configured as members of a group, wherein the steps of said taking a first snapshot, said reconciling, said determining the first difference set, said sending and said applying are performed for each member of the group, the method further comprising:
taking snapshots for implementing remote replication on the first storage system at a first predefined frequency;
defining a first threshold corresponding to resource utilization by the snapshots on the first storage system;
determining when the first threshold has been exceeded on the first storage system; and
in response to determining that the first threshold has been exceeded, reducing the first predefined frequency.

8. The method of claim 7 , further comprising:
defining a second threshold corresponding to the resource utilization, wherein the second threshold is less than the first threshold; and
after the first predefined frequency has been reduced:
determining when the resource utilization has passed below the second threshold, and
in response to determining that the resource utilization has passed below the second threshold, increasing the first predefined frequency.

9. The method of claim 7 , wherein the resource utilization is physical storage consumption by snapshot data of the snapshots on a first storage device.

10. The method of claim 1 , wherein the first logical storage unit is one of one or more logical storage units configured as members of a group, wherein the steps of said taking a first snapshot, said reconciling, said determining the first difference set, said sending and said applying are performed for each member of the group, the method further comprising:
defining a frequency at which snapshots are taken for the first logical storage unit on a first storage d

Assignees

• Emc Ip Holding Co Llc

Inventors

• Bora Bhaskar
• Halstead Mark J
• Vokaliga Deepak
• Yoder Benjamin
• Stronge William R

Classifications

• G06F3/065 Primary

  Replication mechanisms · CPC title

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title