High availability failover manager

Patent metadata

Field	Value
Publication number	US-9785525-B2
Application number	US-201514864026-A
Country	US
Kind code	B2
Filing date	Sep 24, 2015
Priority date	Sep 24, 2015
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

A high availability (HA) failover manager maintains data availability of one or more input/output (I/O) resources in a cluster by ensuring that each I/O resource is available (e.g., mounted) on a hosting node of the cluster and that each I/O resource may be available on one or more partner nodes of the cluster if a node (i.e., a local node) were to fail. The HA failover manager (HA manager) processes inputs from various sources of the cluster to determine whether failover is enabled for a local node and each partner node in an HA group, and for triggering failover of the I/O resources to the partner node as necessary. For each I/O resource, the HA manager may track state information including (i) a state of the I/O resource (e.g., mounted or un-mounted); (ii) the partner node(s) ability to service the I/O resource; and (iii) whether a non-volatile log recording I/O requests is synchronized to the partner node(s). The HA manager interacts with various layers of a storage I/O stack to mount and un-mount the I/O resources on one or more nodes of the cluster through the use of well-defined interfaces, e.g., application programming interfaces.

First claim

What is claimed is:

1. A method comprising:
receiving a write request directed towards a logical unit (LUN), the write request having a data, a logical block address (LBA) and a length representing an address range of the LUN, the LBA and the length mapped to a volume associated with the LUN, the write request received at a first node of a plurality of nodes of a cluster, each node of the cluster having a memory and attached to a storage array storing the volume;
recording the write request in a first non-volatile log of the first node, the first non-volatile log stored on a storage device different from the storage array storing the volume;
monitoring a state of availability of the first node to service the volume;
in response to a lack of availability of the first node to service the volume, determining whether a second node is able to takeover service of the volume; and
in response to determining that the second node is able to takeover service of the volume, triggering a failover of the volume to the second node of the cluster, wherein the first non-volatile log is mirrored to a second non-volatile log accessible by the second node, and wherein the second non-volatile log is up to date with the first non-volatile log.

2. The method of claim 1 further comprising:
recording the state of availability of the first node in a cluster database; and
replicating the cluster database from the first node to the second node of the cluster.

3. The method of claim 2 further comprising:
registering a callback on the second node to monitor a change to the state of availability of the first node recorded in the cluster database; and
receiving a notification at the second node in response to the change of the state of the availability of the first node.

4. The method of claim 2 wherein transactions to the cluster database are ordered and consistent.

5. The method of claim 2 further comprising:
scanning a table of the cluster database at the second node, the table including the state of availability of the first node.

6. The method of claim 1 further comprising:
analyzing heuristics for a plurality of network links used to mirror the first non-volatile log to the second non-volatile log, the plurality of network links connecting the first and second nodes of the cluster; and
configuring the mirror to efficiently use the plurality of network links.

7. The method of claim 1 wherein triggering the failover of the volume further comprises:
mounting the volume on the second node using an operation having no knowledge of the state of availability of the first node to service the volume.

8. The method of claim 1 wherein recording of the state of availability of the first node further comprises using a consensus protocol involving three or more nodes of the cluster.

9. The method of claim 3 further comprising:
determining the state of availability of the first node to service the volume by receiving a notification from the cluster database.

10. A method comprising:
receiving a write request directed towards a logical unit (LUN), the write request having a data, a logical block address (LBA) and a length representing an address range of the LUN, the LBA and the length mapped to a volume associated with the LUN, the write request received at a first node of a plurality of nodes of the cluster, each node of the cluster having a memory and attached to a storage array storing the volume;
recording the write request in a first portion of a non-volatile random access memory (NVRAM) of the first node;
recording a state of availability of the first node to service the volume in a cluster database;
in response to a lack of availability of the first node to service the volume, winning a race at the second node against the first node to update the cluster database to mark the first node as being unavailable to service the volume; and
triggering a failover of the volume to the second node of the cluster, wherein the write request is mirrored to a second portion of the NVRAM accessible by the second node, and wherein the second portion of the NVRAM is up to date with the first portion of the NVRAM.

11. A system comprising:
a cluster having first and second nodes each having a memory connected to a processor via a bus;
a storage array coupled to each node of the cluster;
a storage I/O stack executing on the processor of each node of the cluster, the storage I/O stack configured to:
receive a write request directed towards a logical unit (LUN), the write request having a data, a logical block address (LBA) and a length representing an address range of the LUN, the LBA and the length mapped to a volume associated with the LUN, the write request received at the first node of the cluster, the volume stored on the storage array;
record the write request in a first non-volatile log of the first node, the first non-volatile log stored on a storage device different from the storage array;
monitor a state of availability of the first node to service the volume;
in response to a lack of availability of the first node to service the volume, determine whether the second node is able to takeover service of the volume; and

in response to determining that the second node is able to takeover service of the volume, trigger a failover of the volume to the second node of the cluster, wherein the first non-volatile log is mirrored to a second non-volatile log accessible by the second node, and wherein the second non-volatile log is up to date with the first non-volatile log.

12. The system of claim 11 wherein the storage I/O stack is further configured to:
record the state of availability of the first node in a cluster database; and replicate the cluster database from the first node to the second node of the cluster.

13. The system of claim 12 wherein the storage I/O stack is further configured to:
register a callback on the second node to monitor a change to the state of availability of the first node recorded in the cluster database; and
receive a notification at the second node in response to the change of the state of the availability of the first node.

14. The system of claim 12 wherein transactions to the cluster database are ordered and consistent.

15. The system of claim 12 wherein the storage I/O stack is further configured to:
scan a table of the cluster database at the second node, the table including the state of availability of the first node.

16. The system of claim 11 wherein the storage I/O stack is further configured to:
analyze heuristics for a plurality of network links used to mirror the first non-volatile log to the second non-volatile log, the plurality of network links connecting the first and second nodes of the cluster; and
configure the mirror to efficiently use the plurality of network links.

17. The system of claim 11 wherein the storage I/O stack when configured to trigger a failover over of the volume is further configured to mount the volume on the second node using an operation having no knowledge of the state of availability of the first node to service the volume.

18. The system of claim 11 wherein the storage I/O stack when configured to record the state of availability of the first node is further configured to use a consensus algorithm involving three or more nodes of the cluster.

19. The system of claim 13 wherein the storage I/O stack is further configured to determine the state of availability of first node by receiving a notification from the cluster database.

20. The

Assignees

• Netapp Inc

Inventors

• Watanabe Steven S
• Strange Stephen H
• Muth John
• Malone Kimberly A
• Patel Kayuri H

Classifications

• G06F11/2094 Primary

  Redundant storage or storage space (G06F11/2056 takes precedence) · CPC title

• G06F11/1441

  Resetting or repowering · CPC title

• G06F11/2033

  switching over of hardware resources · CPC title

• G06F11/2046

  where the redundant components share persistent storage (G06F11/2043 takes precedence) · CPC title

• G06F11/2069 Primary

  Management of state, configuration or failover · CPC title