Systems, methods and devices for RDMA read/write operations

Patent metadata

Field	Value
Publication number	US-10257273-B2
Application number	US-201514815662-A
Country	US
Kind code	B2
Filing date	Jul 31, 2015
Priority date	Jul 31, 2015
Publication date	Apr 9, 2019
Grant date	Apr 9, 2019

Abstract

Presented herein are Remote Direct Memory Access (RDMA) networks, RDMA protocols, and methods for performing upper layer protocol (ULP) for RDMA network configurations. Methods for performing ULP RDMA include: a client central processing unit (CPU) allocating a buffer in a memory region of a client computing device; a client network interface card (NIC) creating a steering tag for the buffer; the client CPU transmitting to a local server CPU of a local server node an initiate RDMA request, the RDMA request including source and destination (SRC-DEST) information and a requested RDMA operation, the SRC-DEST information including the steering tag and a file path for a data file; a remote server NIC of a remote server node locating the data file with the file path; the remote server NIC performing the requested RDMA operation without utilizing the server's CPU; and the client and remote server NIC's completing the requested RDMA operation.

First claim

What is claimed:

1. A method, comprising:
receiving a remote direct memory access RDMA request by a remote node, from a local node, via an interconnect connecting the local node and the remote node; wherein the request is generated by a client device for a data file stored by the remote node, the request including a path for the data file, and source and destination information with a steering tag created by a client network interface card (NIC); wherein the client device is connected by a network connection to the local node and the remote node; and
performing an RDMA operation for the data file by a remote node NIC of the remote node, without using a remote CPU, and without routing the data file using the interconnect;
wherein for a read request, the remote node NIC retrieves requested data from a remote node memory and provides the requested directly to the client NIC without using the local node and the remote CPU, and for a write request, data for the data file is received by the remote NIC directly from the client NIC, and written to the remote node memory without using the local node and the remote CPU.

2. The method of claim 1 , further comprising:
receiving an open connection request by the local node from the client device; and
transmitting a connection open success indicator by the local node to the client device.

3. The method of claim 1 , further comprising:
creating a connection between the client device and the remote node, using an RDMA verbs layer, the RDMA verbs layer using a protection domain (PD) to associate a memory region of a buffer of the client device to one or more queue pairs (QP).

4. The method of claim 1 , further comprising:
allocating a holding buffer for the data file, by the remote node CPU in the remote memory; and
registering the holding buffer by the remote node CPU, with the remote node NIC.

5. The method of claim 1 , wherein the steering tag transmitted to the remote node NIC includes information relating to a location of the data file in a data storage module of the remote node.

6. The method of claim 1 , for the read request, further comprising:
reading the data file by the remote node NIC, from a data storage module of the remote node, without utilizing the remote CPU; and
transmitting the data file, by the remote node NIC directly to the client NIC.

7. The method of claim 1 , for the write request, further comprising:
receiving the data for the data file, by the remote node NIC from the client NIC; and
writing the data for the data file, by the remote node NIC without utilizing the remote CPU, to a data storage module of the remote node.

8. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising machine executable code which, when executed by at least one machine, causes the machines to:
receive a remote direct memory access RDMA request by a remote node, from a local node, via an interconnect connecting the local node and the remote node; wherein the request is generated by a client device for a data file stored by the remote nod, the request including a path for the data file, and source and destination information with a steering tag created by a client network interface card (NIC); wherein the client device is connected by a network connection to the local node and the remote node; and
perform an RDMA operation for the data file by a remote node NIC of the remote node, without using a remote CPU, and without routing the data file using the interconnect;
wherein for a read request, the remote node NIC retrieves requested data from a remote node memory and provides the requested directly to the client NIC without using the local node and the remote CPU, and for a write request, data for the data file is received by the remote NIC directly from the client NIC, and written to the remote node memory without using the local node and the remote CPU.

9. The non-transitory machine readable medium of claim 8 , wherein the stored instructions further cause the machines to:
receive an open connection request by the local node from the client device; and
transmit a connection open success indicator by the local node to the client device.

10. The non-transitory machine readable medium of claim 8 , wherein the stored instructions further cause the machines to:
create a connection between the client device and the remote node, using an RDMA verbs layer, the RDMA verbs layer using a protection domain (PD) to associate a memory region of a buffer of the client device to one or more queue pairs (QP).

11. The non-transitory machine readable medium of claim 8 , wherein the stored instructions further cause the machines to:
allocate a holding buffer for the data file, by the remote node CPU in the remote memory; and
register the holding buffer by the remote node CPU, with the remote node NIC.

12. The non-transitory machine readable medium of claim 8 , wherein the steering tag transmitted to the remote node NIC includes information relating to a location of the data file in a data storage module of the remote node.

13. The non-transitory machine readable medium of claim 8 , wherein the stored instructions for the read request, further cause the machines to:
read the data file by the remote node NIC, from a data storage module of the remote node, without utilizing the remote CPU; and
transmit the data file, by the remote node NIC directly to the client NIC.

14. The non-transitory machine readable medium of claim 8 , wherein the stored instructions for the write request, further cause the machines to:
receive the data for the data file, by the remote node NIC from the client NIC; and
write the data for the data file, by the remote node NIC without utilizing the remote CPU, to a data storage module of the remote node.

15. A system comprising:
a memory containing machine readable medium comprising machine executable code having stored thereon instructions; and a processor configured to execute the machine executable code to:
receive a remote direct memory access RDMA request by a remote node, from a local node, via an interconnect connecting the local node and the remote node; wherein the request is generated by a client device for a data file stored by the remote node, the request including a path for the data file, and source and destination information with a steering tag created by a client network interface card (NIC); wherein the client device is connected by a network connection to the local node and the remote node; and
perform an RDMA operation for the data file by a remote node NIC of the remote node, without using a remote CPU, and without routing the data file using the interconnect;
wherein for a read request, the remote node NIC retrieves requested data from a remote node memory and provides the requested directly to the client NIC without using the local node and the remote CPU, and for a write request, data for the data file is received by the remote NIC directly from the client NIC, and written to the remote node memory without using the local node and the remote CPU.

16. The system of claim 15 , wherein the stored instructions further cause the executable code to:
receive an open connection request by the local node from the client device; and
transmit a connection open success indicator by the local node to the client device.

17. The system of claim 15 , wherein the stored instructions further causes the executable code to:
create a connection between the client device and the remote node, using an RDMA verbs layer, the RDMA verbs layer usi

Assignees

• Netapp Inc

Inventors

• Govind Subin

Classifications

• H04L67/1008

  based on parameters of servers, e.g. available memory or workload (monitoring of computer activity G06F11/30) · CPC title

• G06F15/17331

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title

• H04L67/1097 Primary

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F16/172

  Caching, prefetching or hoarding of files · CPC title

• G06F16/00

  Information retrieval; Database structures therefor; File system structures therefor · CPC title