Storage system and server

Patent metadata

Field	Value
Publication number	US-9727503-B2
Application number	US-201414215099-A
Country	US
Kind code	B2
Filing date	Mar 17, 2014
Priority date	Mar 17, 2014
Publication date	Aug 8, 2017
Grant date	Aug 8, 2017

Abstract

A data storage system includes a storage server, including non-volatile memory (NVM) and a server network interface controller (NIC), which couples the storage server to a network. A host computer includes a host central processing unit (CPU), a host memory and a host NIC, which couples the host computer to the network. The host computer runs a driver program that is configured to receive, from processes running on the host computer, commands in accordance with a protocol defined for accessing local storage devices connected to a peripheral component interface bus of the host computer, and upon receiving a storage access command in accordance with the protocol, to initiate a remote direct memory access (RDMA) operation to be performed by the host and server NICs so as to execute on the storage server, via the network, a storage transaction specified by the command.

First claim

The invention claimed is:

1. A host computer for accessing a data storage system, comprising:
a host memory;
a host central processing unit (CPU);
a peripheral component interface bus connecting the host CPU to peripheral components; and
a host network interface controller (NIC) configured to expose both a storage interface and a network communication interface to the peripheral component interface bus, and is configured to receive, through the storage interface, from processes running on the CPU of the host computer, commands in accordance with a protocol defined for accessing local storage devices connected to the peripheral component interface bus, and upon receiving a storage access command in accordance with the protocol defined for accessing local storage devices, to initiate a remote direct memory access (RDMA) operation instructing a storage server, via the network, to execute a storage transaction specified by the command in accordance with the protocol defined for accessing local storage devices,
wherein the protocol is an NVM Express protocol, and the storage server comprises a solid-state drive (SSD),
wherein the storage server is configured, after committing the storage transaction, to cause a server NIC of the storage server to transmit a completion notification via the network to a completion queue in the host memory,
wherein the server NIC is configured to detect a completion entry posted by a controller of the storage server, and to transmit the completion notification, in response to the detected completion entry, via the network to the host NIC, and
wherein each of the storage interface and the network communication interface occupies a corresponding address range on the peripheral component interface bus, and wherein the corresponding address ranges of the storage interface and the network communication interface do not include a common range.

2. The host computer according to claim 1 , wherein the host CPU is configured to place the commands in a submission queue in the host memory and the host NIC is configured to write a mirror of the submission queue, via the network, to a server memory of the storage server.

3. The host computer according to claim 2 , wherein the host NIC is configured to additionally initiate another RDMA operation, wherein in the another RDMA operation the host NIC writes an entry via the network to a doorbell register of the storage server, to cause the storage server to read the storage access command from the mirror of the submission queue and carry out the storage transaction responsively to the storage access command.

4. The host computer according to claim 2 , wherein the storage access command comprises a write command to be carried out by the host computer, and wherein the host computer is configured to select a buffer in the server memory and to incorporate a pointer to the buffer in the write command that is posted in the mirror of the submission queue, and to cause the host NIC to perform an RDMA write operation to transfer data to the selected buffer.

5. The host computer according to claim 1 , wherein the storage access command comprises a read command to be carried out by the host computer, and wherein the host NIC is configured to convey the read command in the RDMA operation to the storage server, causing a server NIC of the storage server to initiate an RDMA write operation to be performed by the server NIC and the host NIC, to the host memory.

6. The host computer according to claim 1 , wherein the storage access command comprises a write command to be carried out by the host computer, and wherein the host NIC is configured to convey the write command in the RDMA operation to the storage server, causing a server NIC of the storage server to initiate an RDMA read operation to be performed by the server NIC and the host NIC from the host memory.

7. The host computer according to claim 1 , wherein the storage access command comprises a scatter/gather list specifying a non-contiguous set of memory ranges in the host memory, and wherein the host NIC is configured to map the memory ranges to a contiguous range of virtual memory and to cause the storage server to execute the storage transaction with reference to the contiguous range.

8. The host computer according to claim 7 , wherein a server NIC is configured to execute the storage transaction by performing the RDMA operation with reference to the contiguous range of the virtual memory.

9. The host computer according to claim 7 , wherein the storage server has a server memory, and wherein the host computer is configured to map the contiguous range of virtual memory to a buffer in the server memory, for use by the storage server in the storage transaction.

10. The host computer according to claim 1 , wherein the host NIC is configured to write, in a further RDMA operation, to a completion doorbell of the storage server, responsive to the completion notification indicating the storage access command was completed.

11. A host network interface controller (NIC), comprising:
a network interface configured to be connected to a network and to convey data packets, via the network, to and from a storage server on the network;
a first host interface configured to be connected to a peripheral component interface bus of a host computer;
a second host interface configured to be connected to the peripheral component interface bus, while exposing a storage interface on the peripheral component interface bus; and
processing circuitry configured to receive, via the second host interface, commands in accordance with a protocol defined for accessing local storage devices connected to the peripheral component interface bus of the host computer, and upon receiving a storage access command in accordance with the protocol defined for accessing local storage devices, to initiate a remote direct memory access (RDMA) operation to be performed via the network interface, wherein the RDMA operation causes the storage server to execute, via the network, a storage transaction specified by the storage access command in accordance with the protocol defined for accessing local storage devices,
wherein the protocol is an NVM Express protocol, and the storage server comprises a solid-state drive (SSD),
wherein the storage server is configured, after committing the storage transaction, to cause a server NIC of the storage server to transmit a completion notification via the network to a completion queue in the host memory,
wherein the server NIC is configured to detect a completion entry posted by a controller of the storage server, and to transmit the completion notification, in response to the detected completion entry, via the network to the host NIC, and
wherein each of the first and second host interfaces occupies a corresponding address range on the peripheral component interface bus, and wherein the corresponding address ranges of the first and second host interfaces do not include a common range.

Assignees

• Mellanox Technologies Ltd
• Mellanox Technologies Ltd

Inventors

• Kagan Michael
• Bloch Noam
• Raikin Shlomo
• Haviv Yaron
• Burstein Idan

Classifications

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/0661

  Format or protocol conversion arrangements · CPC title

• G06F13/28 Primary

  using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) · CPC title