Reducing redundancy in synchronous replication of metadata with a journal log

Patent metadata

Field	Value
Publication number	US-12099477-B2
Application number	US-202117510551-A
Country	US
Kind code	B2
Filing date	Oct 26, 2021
Priority date	Oct 26, 2021
Publication date	Sep 24, 2024
Grant date	Sep 24, 2024

Abstract

A method of performing synchronous replication from a primary storage system apparatus (PSSA) to a secondary storage system apparatus (SSSA) is provided. The method includes (a) in response to write requests received by the PSSA, (i) calculating metadata changes by the PSSA for accommodating the write requests, (ii) generating, by the PSSA, metadata journal log entries that describe the metadata changes, and (iii) mirroring the metadata journal log entries from the PSSA to the SSSA; (b) regenerating the metadata changes by the SSSA based on the metadata journal log entries mirrored from the PSSA to the SSSA; and (c) writing the regenerated metadata changes to persistent storage of the SSSA. A method performed by the SSSA is also provided. An apparatus, system, and computer program product for performing similar methods are also provided.

First claim

What is claimed is:

1. A method of performing synchronous replication from a primary storage system apparatus (PSSA) to a secondary storage system apparatus (SSSA), the method comprising:
in response to write requests received by the PSSA and directed to a filesystem on the PSSA, (i) calculating metadata changes by the PSSA for accommodating the write requests, the metadata changes including changes in a plurality of metadata blocks of the filesystem, (ii) generating, by the PSSA, metadata journal log entries that describe the metadata changes, and (iii) mirroring the metadata journal log entries but not the plurality of metadata blocks from the PSSA to the SSSA;
regenerating the metadata changes by the SSSA based on the metadata journal log entries mirrored from the PSSA to the SSSA; and
writing the regenerated metadata changes to persistent storage of the SSSA,
wherein the metadata journal log entries generated by the PSSA are smaller than the plurality of metadata blocks.

2. The method of claim 1 wherein regenerating the metadata changes includes the SSSA playing metadata changes described by the metadata journal log entries into a metadata buffer.

3. The method of claim 2 wherein regenerating the metadata changes further includes the SSSA reading preexisting metadata into the metadata buffer.

4. The method of claim 3 wherein:
playing the metadata changes is performed in response to the metadata journal log entries being mirrored from the PSSA to the SSSA;
reading the preexisting metadata into the metadata buffer is performed in response to receiving a metadata write command shell from the PSSA, the metadata write command shell being received from the PSSA after the metadata journal log entries are mirrored from the PSSA to the SSSA; and
writing the regenerated metadata changes includes writing the metadata buffer to the persistent storage of the SSSA after playing the metadata changes and reading the preexisting metadata into the metadata buffer.

5. The method of claim 4 wherein:
the metadata changes include changes to an upper-level filesystem, the upper-level filesystem having a first address space, the upper-level filesystem being stored on a virtual volume backed by a lower-level filesystem, the lower-level filesystem having a second address space distinct from the first address space; and
the metadata write command shell includes an address within the second address space to which the metadata changes are to be written.

6. The method of claim 3 wherein each of the metadata journal log entries includes a tag directing that the metadata changes described by that metadata journal log entry should be kept in cache until those metadata changes can be played into the metadata buffer.

7. The method of claim 3 wherein playing metadata changes described by the metadata journal log entries into the metadata buffer includes:
playing a first set of metadata changes described by a first metadata journal log entry into the metadata buffer;
subsequently, playing a second set of metadata changes described by a second metadata journal log entry into the metadata buffer, the second metadata journal log entry having been mirrored to the SSSA after the first metadata journal log entry, wherein playing the second set of metadata changes includes overwriting a metadata change played into the metadata buffer that was described by the first metadata journal log entry; and
writing the regenerated metadata changes includes writing the metadata buffer to the persistent storage of the SSSA after (a) playing the first set of metadata changes and the second set of metadata changes and (b) reading the preexisting metadata into the metadata buffer.

8. The method of claim 3 wherein:
playing the metadata changes and reading the preexisting metadata into the metadata buffer are performed in response to the metadata journal log entries being mirrored from the PSSA to the SSSA; and
writing the regenerated metadata changes includes writing the metadata buffer to the persistent storage of the SSSA in response to playing the metadata changes and reading the preexisting metadata into the metadata buffer.

9. A method performed by a secondary storage system apparatus (SSSA) of performing synchronous replication from a primary storage system apparatus (PSSA), the method comprising:
receiving mirrored metadata journal log entries from the PSSA, the mirrored metadata journal log entries having been generated by the PSSA in response to write requests received by the PSSA and directed to a filesystem on the PSSA by (i) calculating metadata changes for accommodating the write requests, the metadata changes including changes in a plurality of metadata blocks of the filesystem, (ii) generating the metadata journal log entries to describe the metadata changes, and (iii) mirroring the metadata journal log entries but not the plurality of metadata blocks from the PSSA to the SSSA;
regenerating the metadata changes based on the received metadata journal log entries mirrored from the PSSA to the SSSA; and
writing the regenerated metadata changes to persistent storage of the SSSA,
wherein the metadata journal log entries generated by the PSSA are smaller than the plurality of metadata blocks.

10. The method of claim 9 wherein regenerating the metadata changes includes the SSSA playing metadata changes described by the metadata journal log entries into a metadata buffer.

11. The method of claim 10 wherein regenerating the metadata changes further includes the SSSA reading preexisting metadata into the metadata buffer.

12. The method of claim 11 wherein:
playing the metadata changes is performed in response to the metadata journal log entries being mirrored from the PSSA to the SSSA;
reading the preexisting metadata into the metadata buffer is performed in response to receiving a metadata write command shell from the PSSA, the metadata write command shell being received from the PSSA after the metadata journal log entries are mirrored from the PSSA to the SSSA; and
writing the regenerated metadata changes includes writing the metadata buffer to the persistent storage of the SSSA after playing the metadata changes and reading the preexisting metadata into the metadata buffer.

13. The method of claim 12 wherein:
the metadata changes include changes to an upper-level filesystem, the upper-level filesystem having a first address space, the upper-level filesystem being stored on a virtual volume backed by a lower-level filesystem, the lower-level filesystem having a second address space distinct from the first address space; and
the metadata write command shell includes an address within the second address space to which the metadata changes are to be written.

14. The method of claim 11 wherein each of the metadata journal log entries includes a tag directing that the metadata changes described by that metadata journal log entry should be kept in cache until those metadata changes can be played into the metadata buffer.

15. The method of claim 11 wherein playing metadata changes described by the metadata journal log entries into the metadata buffer includes:
playing a first set of metadata changes described by a first metadata journal log entry into the metadata buffer;
subsequently, playing a second set of metadata changes described by a second metadata journal log entry into the metadata buffer, the second metadata journal log entry having been mirrored to the SSSA after the first metadata journal log entry, wherein playing the second set of metadata changes includes overwriting a metadata change played into the metadata buffer th

Assignees

• Emc Ip Holding Co Llc

Inventors

• Gu Joseph Yuan
• Xiong Tianfang
• Qu Qi
• Zhang Weibing
• Gao Lei
• Yim Wai C
• Gong Chen

Classifications

• G06F16/1805

  Append-only file systems, e.g. using logs or journals to store data · CPC title

• G06F16/164

  File meta data generation · CPC title

• G06F16/172

  Caching, prefetching or hoarding of files · CPC title

• H04L67/025

  for remote control or remote monitoring of applications · CPC title

• G06F3/0683

  Plurality of storage devices · CPC title