File-based cluster-to-cluster replication recovery

Patent metadata

Field	Value
Publication number	US-9697092-B2
Application number	US-201514836999-A
Country	US
Kind code	B2
Filing date	Aug 27, 2015
Priority date	Aug 27, 2015
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

Resiliency in file replication scenarios via maintenance of multiple replication queues with differing susceptibility to various sources of data loss. A plurality of requests are received at a first file system location to write to one or more file system objects at the first file system location. Details about the plurality of requests are recorded in at least two replication queues which differ, at least in part, as to types of failures which will cause data loss. Responsive to detection of data loss in one of the replication queues, details about the plurality of requests are retrieved from another of the replication queues.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, at a first file system location, a plurality of requests to write to one or more file system objects at the first file system location;
recording details about the plurality of requests in both a primary and a secondary replication queue; and
responsive to detection of data loss in the primary replication queue, retrieving the details about the plurality of requests from the secondary replication queue;
wherein:
both the primary and the secondary replication queues are at the first file system location; and
the primary and secondary queues differ, at least in part, as to types of failures which will cause data loss.

2. The computer-implemented method of claim 1 further comprising:
responsive to receiving the plurality of requests, writing to a first copy of the one or more file system objects at the first file system location; and
responsive to the retrieved details, replicating results of the plurality of write requests to a second file system location by writing to a second copy of the one or more file system objects at the second file system location.

3. The computer-implemented method of claim 2 wherein:
the details recorded in the secondary replication queue identify which file system object(s) are affected by the plurality of requests; and
the results of the plurality of write requests are replicated to the second file system location by transferring the entirety of the file system objects affected by the plurality of write requests from the first file system location to the second file system location.

4. The computer-implemented method of claim 2 wherein:
the details recorded in the secondary replication queue include change data for the file system objects affected by the plurality of write requests; and
the results of the plurality of write requests are replicated to the second file system location by transferring only changed portions of the file system objects affected by the plurality of write requests from the first file system location to the second file system location.

5. The computer-implemented method of claim 1 wherein the details recorded in the secondary replication queue survive a restart of the first file system location.

6. The computer-implemented method of claim 1 wherein:
the primary replication queue is an in-memory data structure stored in volatile storage at the first file system location; and
the secondary replication queue is an inode stored in non-volatile storage at the first file system location.

7. The computer-implemented method of claim 1 wherein the first file system location is a first node of a clustered file system and the second file system location is a second node of the clustered file system.

8. A computer program product comprising a non-transitory computer readable storage medium having stored thereon:
first instructions executable by a device to cause the device to receive, at a first file system location, a plurality of requests to write to one or more file system objects at the first file system location;
second instructions executable by the device to cause the device to record details about the plurality of requests in both a primary and a secondary replication queue; and
third instructions executable by the device to cause the device to retrieve, responsive to detection of data loss in the primary replication queue, the details about the plurality of requests from the secondary replication queue;
wherein:
both the primary and the secondary replication queues are at the first file system location; and
the primary and secondary queues differ, at least in part, as to types of failures which will cause data loss.

9. The product of claim 8 wherein the non-transitory computer readable storage medium has further stored thereon:
fourth instructions executable by the device to cause the device to write, responsive to receiving the plurality of requests, to a first copy of the one or more file system objects at the first file system location; and
fifth instructions executable by the device to cause the device to replicate, responsive to the retrieved details, results of the plurality of write requests to a second file system location by writing to a second copy of the one or more file system objects at the second file system location.

10. The product of claim 9 wherein:
the details recorded in the secondary replication queue include change data for the file system objects affected by the plurality of write requests; and
the results of the plurality of write requests are replicated to the second file system location by transferring only changed portions of the file system objects affected by the plurality of write requests from the first file system location to the second file system location.

11. The product of claim 8 wherein the details recorded in the secondary replication queue survive a restart of the first file system location.

12. The product of claim 8 wherein:
the primary replication queue is an in-memory data structure stored in volatile storage at the first file system location; and
the secondary replication queue is an inode stored in non-volatile storage at the first file system location.

13. The product of claim 8 wherein the first file system location is a first node of a clustered file system and the second file system location is a second node of the clustered file system.

14. A computer system comprising:
a processor(s) set; and
a computer readable storage medium;
wherein:
the processor set is structured, located, connected and/or programmed to execute instructions stored on the computer readable storage medium;
the instructions include:
first instructions executable by a device to cause the device to receive, at a first file system location, a plurality of requests to write to one or more file system objects at the first file system location;
second instructions executable by the device to cause the device to record details about the plurality of requests in both a primary and a secondary replication queue; and
third instructions executable by the device to cause the device to retrieve, responsive to detection of data loss in the primary replication queue, the details about the plurality of requests from the secondary replication queue;

both the primary and the secondary replication queues are at the first file system location; and
the primary and secondary queues differ, at least in part, as to types of failures which will cause data loss.

15. The system of claim 14 wherein the instructions further include:
fourth instructions executable by the device to cause the device to write, responsive to receiving the plurality of requests, to a first copy of the one or more file system objects at the first file system location; and
fifth instructions executable by the device to cause the device to replicate, responsive to the retrieved details, results of the plurality of write requests to a second file system location by writing to a second copy of the one or more file system objects at the second file system location.

16. The system of claim 15 wherein:
the details recorded in the secondary replication queue identify which file system object(s) are affected by the plurality of requests; and
the results of the plurality of write requests are replicated to the second file system location by transferring the entirety of the file system objects affected by the plurality of write requests from the first file system location to the second file system location.

17. The system of

Assignees

• IBM

Inventors

• Dave Abhishek S
• Singh Ajay

Classifications

• G06F11/1662

  the resynchronized component or unit being a persistent storage device (re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008) · CPC title

• G06F2201/80

  Database-specific techniques · CPC title

• G06F11/2094

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/1441

  Resetting or repowering · CPC title

• G06F11/1471 Primary

  involving logging of persistent data for recovery · CPC title