Failover handling in modular switched fabric for data storage systems

Patent metadata

Field	Value
Publication number	US-10496504-B2
Application number	US-201715625216-A
Country	US
Kind code	B2
Filing date	Jun 16, 2017
Priority date	Jun 23, 2014
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

Data storage systems are provided that include network interfaces each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric. The data storage systems also include processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transfer ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric. The data storage systems also identifie failed ones of the processors and responsively initiate backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors.

First claim

What is claimed is:

1. A data storage system, comprising:
a plurality of network interfaces each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric;
a plurality of processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transfer ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric, wherein the shared PCIe address space is shared among at least the plurality of processors on the PCIe fabric; and
a monitoring system configured to identify failed ones of the processors and responsively initiate backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors.

2. The data storage system of claim 1 , comprising:
the plurality of processors configured to maintain failover tables comprising indications of the allocation information of the shared PCIe address space among the plurality of processors.

3. The data storage system of claim 1 , comprising:
the plurality of processors comprising the monitoring system, and configured to periodically poll one or more monitored processors to identify failed ones of the processors.

4. The data storage system of claim 1 , comprising:
a control system configured to subdivide the shared PCIe address space to allocate a plurality of storage drives among individual processors, wherein each subdivision of the shared PCIe address space corresponds to an associated subset of the plurality of storage drives.

5. The data storage system of claim 1 , comprising:
a control system configured to assign each of a plurality of storage drives an associated unique identifier and report the associated unique identifiers to the processors for handling of the storage operations over the PCIe fabric.

6. The data storage system of claim 1 , comprising:
a control system configured to initialize a plurality of storage drives for use on the PCIe fabric by the plurality of processors;
the control system configured to assign each of the plurality of storage drives an associated unique identifier using at least a sideband communication link distinct from the PCIe fabric, the sideband communication link communicatively coupling the control system and the plurality of storage drives; and
the control system configured to report the associated unique identifiers to the plurality of processors.

7. The data storage system of claim 1 , comprising:
responsive to a failed processor, a monitoring processor configured to inherit network connection states from the failed processor and accept ones of the storage operations directed to a network interface initially assigned to the failed processor.

8. The data storage system of claim 1 , comprising:
responsive to a failed processor, a monitoring processor configured to initiate a rediscovery process over the PCIe fabric with one or more storage drives formerly managed by the failed processor to transfer management to the monitoring processor.

9. The data storage system of claim 1 , comprising:
responsive to detecting the failed ones of the processors, the monitoring system configured to restart the failed ones of the processors.

10. A method of operating a data storage system, the method comprising:
in a plurality of network interfaces, receiving storage operations issued by one or more host systems for handling by the data storage system, and transferring the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric;
in a plurality of processors, receiving the storage operations over the PCIe fabric, processing properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operations, and selectively transferring ones of the storage operations to the target processors for handling of the storage operations with associated storage drives coupled over the PCIe fabric, wherein the shared PCIe address space is shared among at least the plurality of processors on the PCIe fabric; and
in a monitoring system, identifying failed ones of the processors and responsively initiating backup ones of the processors to handle portions of the shared PCIe address space handled by the failed ones of the processors.

11. The method of claim 10 , further comprising:
in the plurality of processors, maintaining failover tables comprising indications of the allocation information of the shared PCIe address space among the plurality of processors.

12. The method of claim 10 , further comprising:
the plurality of processors, periodically polling one or more monitored processors to identify failed ones of the processors.

13. The method of claim 10 , further comprising:
in a control system, subdividing the shared PCIe address space to allocate a plurality of storage drives among individual processors, wherein each subdivision of the shared PCIe address space corresponds to an associated subset of the plurality of storage drives.

14. The method of claim 10 , further comprising:
in a control system, assigning each of a plurality of storage drives an associated unique identifier, and reporting the associated unique identifiers to the processors for handling of the storage operations over the PCIe fabric.

15. The method of claim 10 , further comprising:
in a control system, initializing a plurality of storage drives for use on the PCIe fabric by the processors, and assigning each of the plurality of storage drives an associated unique identifier using at least a sideband communication link distinct from the PCIe fabric, the sideband communication link communicatively coupling the control system and the plurality of storage drives; and
in the control system, reporting the associated unique identifiers to the plurality of processors.

16. The method of claim 10 , further comprising:
responsive to a failed processor, in a monitoring processor, inheriting network connection states from the failed processor and accepting ones of the storage operations directed to a network interface initially assigned to the failed processor.

17. The method of claim 10 , further comprising:
responsive to a failed processor, in a monitoring processor, initiating a rediscovery process over the PCIe fabric with one or more storage drives formerly managed by the failed processor to transfer management to the monitoring processor.

18. The method of claim 10 , further comprising:
responsive to detecting the failed ones of the processors, restarting the failed ones of the processors.

19. A modular data storage system, comprising:
a plurality of network modules each configured to receive storage operations issued by one or more host systems for handling by the data storage system, and transfer the storage operations over a Peripheral Component Interconnect Express (PCIe) fabric;
a plurality of processing modules each comprising one or more processors configured to receive the storage operations over the PCIe fabric, process properties of the storage operations against allocation information of a shared PCIe address space to determine target processors to handle the storage operati

Assignees

• Liqid Inc

Inventors

• Breakstone Jason
• Long Christopher R
• Cannata James Scott

Classifications

• G06F11/2038

  with a single idle spare processing component · CPC title

• G06F11/2046

  where the redundant components share persistent storage (G06F11/2043 takes precedence) · CPC title

• G06F2213/0026

  PCI express · CPC title

• G06F3/0607

  by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title