Priming failover of stateful offload adapters

Patent metadata

Field	Value
Publication number	US-9442812-B2
Application number	US-201414303720-A
Country	US
Kind code	B2
Filing date	Jun 13, 2014
Priority date	Dec 12, 2013
Publication date	Sep 13, 2016
Grant date	Sep 13, 2016

Abstract

Mechanisms, in a data processing system comprising a first adapter and second adapter, for performing a failover operation from the first adapter to the second adapter are provided. The mechanisms detect that an imminent failure of the first adapter is likely to occur and initiate a failover priming operation in the first adapter and second adapter in response to detecting the imminent failure. The failover priming operation configures ingress and egress buffers of the second adapter to have a similar configuration to ingress and egress buffers of the first adapter. The mechanisms migrate processing of ingress data traffic to the second adapter prior to failure of the first adapter such that the first adapter processes egress data traffic from the data processing system and the second adapter processes ingress data traffic to the data processing system.

First claim

What is claimed is:

1. A method, in a data processing system comprising a first adapter and second adapter, for performing a failover operation from the first adapter to the second adapter, the method comprising:
detecting that an imminent failure of the first adapter is likely to occur;
initiating a failover priming operation in the first adapter and second adapter in response to detecting the imminent failure, wherein the failover priming operation configures ingress and egress buffers of the second adapter to have a similar configuration to ingress and egress buffers of the first adapter; and
migrating processing of ingress data traffic to the second adapter prior to failure of the first adapter such that the first adapter processes egress data traffic from the data processing system and the second adapter processes ingress data traffic to the data processing system.

2. The method of claim 1 , further comprising:
detecting an actual failure of the first adapter subsequent to detecting that the imminent failure of the first adapter is likely to occur; and
in response to detecting the actual failure of the first adapter, migrating processing of egress data traffic from the first adapter to the second adapter.

3. The method of claim 1 , wherein initiating the failover priming operation comprises:
copying direct memory access (DMA) queue and buffer configuration and state information of the first adapter to the second adapter; and
configuring DMA queues and the ingress and egress buffers of the second adapter using the DMA queue and buffer configuration and state information of the first adapter.

4. The method of claim 1 , wherein migrating processing of ingress data traffic to the second adapter prior to failure of the first adapter comprises:
pausing ingress data traffic processing on the first adapter;
calculating a size of data currently stored in an ingress buffer of the first adapter;
calculating a skip position in an ingress direct memory access (DMA) queue associated with the first adapter based on the size of data currently stored in the ingress buffer of the first adapter; and
communicating the skip position to the second adapter.

5. The method of claim 4 , wherein migrating processing of ingress data traffic to the second adapter prior to failure of the first adapter further comprises:
re-enabling processing of data in the ingress buffer of the first adapter; and
processing data in the ingress buffer of the first adapter down to the calculated skip position to thereby drain the ingress buffer of the first adapter.

6. The method of claim 5 , wherein migrating processing of the ingress data traffic to the second adapter further comprises:
programming an ingress DMA queue current position of an ingress DMA queue of the second adapter based on the skip position; and
enabling processing of ingress data traffic on the second adapter, wherein processing of the ingress data traffic on the second adapter starts at the ingress DMA queue current position.

7. The method of claim 6 , wherein processing of the ingress data traffic on the second adapter is enabled without enabling interrupts on the second adapter, and wherein interrupts are enabled on the second adapter in response to the ingress buffer of the first adapter becoming empty.

8. The method of claim 7 , further comprising enabling an ingress buffer and ingress physical port of the second adapter in response to enabling interrupts on the second adapter.

9. The method of claim 1 , further comprising:
in response to migrating processing of the ingress data traffic to the second adapter, determining if egress data is flowing through the first adapter; and
in response to a determination that egress data traffic is not flowing through the first adapter, migrating processing of the egress data traffic from the first adapter to the second adapter.

10. The method of claim 1 , further comprising:
determining whether a performance of the first adapter continues to deteriorate from a performance state when the imminent failure of the first adapter was detected; and
in response to determining that the performance of the first adapter continues to deteriorate:
pausing egress DMA transactions on the first adapter;
communicating a position in an egress DMA queue of the first adapter to the second adapter;
enabling an egress function of the second adapter to begin reading data at the position in the egress DMA queue of the first adapter;
draining the egress DMA queue of the first adapter; and
enabling egress data traffic processing by the second adapter in response to draining the egress DMA queue of the first adapter.

Assignees

• IBM

Inventors

• Cardona Omar
• Ochs Matthew R
• Sethi Vikramjit

Classifications

• G06F11/2033

  switching over of hardware resources · CPC title

• G06F11/2005

  using redundant communication controllers · CPC title

• G06F11/1658 Primary

  Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit · CPC title

• G06F15/17331

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title

• G06F11/203 Primary

  using migration · CPC title