Switchless fabric based atomics via partial-proxy

Patent metadata

Field	Value
Publication number	US-9910753-B1
Application number	US-201514974286-A
Country	US
Kind code	B1
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A data storage system has first and second computing nodes that are interconnected by a switchless fabric. Each storage node includes first and second paired storage directors with an interconnecting communication link. Atomic operations sent between the computing nodes are mediated by network adapters. Atomic operations sent between paired storage directors via the interconnecting communication link are provided to a network adapter via an internal port and mediated by network adapter. The interconnecting communication links can be used as a backup path for atomic operations in the event of a link failure of the switchless fabric.

First claim

What is claimed is:

1. An apparatus comprising:
a data storage system comprising:
a plurality of data storage devices;
a first computing node comprising paired first and second storage directors and a communications link between the paired first and second storage directors, each of the paired first and second storage directors comprising a processor, a memory, an interface to at least some of the data storage devices, and a network adapter that mediates atomic operations;
a second computing node comprising paired third and fourth storage directors and a communications link between the paired third and fourth storage directors, each of the paired third and fourth storage directors comprising a processor, a memory, an interface to at least some of the data storage devices, and a network adapter that mediates atomic operations; and
a switchless fabric that interconnects the first computing node with the second computing node via the network adapters;
wherein an atomic operation provided by the first storage director is sent as a message to the second storage director via the communications link between the paired first and second storage directors and mediated by the network adapter of the second storage director.

2. The apparatus of claim 1 wherein the switchless fabric comprises multiple point-to-point links between physical ports of the network adapters of the storage directors.

3. The apparatus of claim 1 wherein the atomic operation is used to perform direct memory access between the paired first and second storage directors.

4. The apparatus of claim 1 wherein the network adapters of the first computing node are directly connected to the network adapters of the second computing node.

5. The apparatus of claim 1 wherein each network adapter performs mediation by serializing atomic operations and implementing locks on the memory of the storage director with which the respective network adapter is associated.

6. The apparatus of claim 1 wherein the atomic operation is an atomic read and the network adapter of the second storage director obtains data at a location of the memory of the second storage director indicated by the atomic read.

7. The apparatus of claim 1 wherein the atomic operation is an atomic write and the network adapter of the second storage director establishes a lock at a location of the memory of the second storage director indicated by the atomic write, writes data to the location, and returns an acknowledgement to the first storage director.

8. The apparatus of claim 1 wherein provision of the atomic operation by the first storage director to the second storage director is in response to detection of a failed link of the switchless fabric.

9. The apparatus of claim 1 wherein the atomic operation is initiated by the third storage director and provided by the third storage director to the first storage director via the switchless fabric.

10. The apparatus of claim 9 wherein a link of the switchless fabric between the third storage director and the second storage director is failed, and provision of the atomic operation by the third storage director to the first storage director via the switchless fabric is in response to detection of the failed link.

11. A method comprising:
in a data storage system comprising: a plurality of data storage devices; a first computing node comprising paired first and second storage directors and a communications link between the paired first and second storage directors, each of the paired first and second storage directors comprising a processor, a memory, an interface to at least some of the data storage devices, and a network adapter; a second computing node comprising paired third and fourth storage directors and a communications link between the paired third and fourth storage directors, each of the paired third and fourth storage directors comprising a processor, a memory, an interface to at least some of the data storage devices, and a network adapter; and a switchless fabric that interconnects the first computing node with the second computing node via the network adapters:
providing an atomic operation by the first storage director as a message to the second storage director via the communications link between the paired first and second storage directors; and
mediating the atomic operation with the network adapter of the second storage director.

12. The method of claim 11 wherein the switchless fabric comprises multiple point-to-point links between physical ports of the network adapters of the storage directors, and comprising sending atomic operations via the point-to-point links.

13. The method of claim 11 comprising using the atomic operation to perform direct memory access between the paired first and second storage directors.

14. The method of claim 11 wherein the network adapters of the first computing node are directly connected to the network adapters of the second computing node and comprising sending atomic operations between a network adapter of the first computing node and a network adapter of the second computing node.

15. The method of claim 11 comprising each network adapter performing mediation by serializing atomic operations and implementing locks on the memory of the storage director with which the respective network adapter is associated.

16. The method of claim 11 wherein the atomic operation is an atomic read and comprising the network adapter of the second storage director obtaining data at a location of the memory of the second storage director indicated by the atomic read.

17. The method of claim 11 wherein the atomic operation is an atomic write and comprising the network adapter of the second storage director establishing a lock at a location of the memory of the second storage director indicated by the atomic write, writing data to the location, and returning an acknowledgement to the first storage director.

18. The method of claim 11 comprising providing the atomic operation by the first storage director to the second storage director in response to detection of a failed link of the switchless fabric.

19. The method of claim 11 comprising initiating the atomic operation by the third storage director and providing the atomic operation by the third storage director to the first storage director via the switchless fabric.

20. The method of claim 19 wherein a link of the switchless fabric between the third storage director and the second storage director is failed, and comprising providing the atomic operation by the third storage director to the first storage director via the switchless fabric is in response to detection of the failed link.

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Tringale Alesia
• Mcclure Steven T
• Cartmell Jerome
• Zhivich Julie

Classifications

• G06F11/221

  to test buses, lines or interfaces, e.g. stuck-at or open line faults · CPC title

• G06F13/28 Primary

  using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) · CPC title

• G06F11/2012

  and using different communication protocols · CPC title

• G06F3/0607

  by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device · CPC title

• G06F11/3034

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title