Providing remote, reliant and high performance PCI express device in cloud computing environments

Patent metadata

Field	Value
Publication number	US-10936535-B2
Application number	US-201916359418-A
Country	US
Kind code	B2
Filing date	Mar 20, 2019
Priority date	Nov 21, 2014
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

A system architecture, a method, and a computer program product are disclosed for attaching remote physical devices. In one embodiment, the system architecture comprises a compute server and a device server. The compute server includes a system memory, and one or more remote device drivers; and the device server includes a system memory and one or more physical devices, and each of the physical devices includes an associated device memory. The compute server and the device server are connected through an existing network fabric that provides remote direct memory access (RDMA) services. A system mapping function logically connects one or more of the physical devices on the device server to the compute server, including mapping between the system memories and the device memories and keeping the system memories and the device memories in synchronization using the RDMA.

First claim

The invention claimed is:

1. A method for attaching remote physical devices of a device server to a compute server through an existing network fabric providing remote direct memory access (RDMA) services, the compute server including a compute server system memory, and running an operating system and one or more remote device drivers; the device server including a device server system memory and one or more physical devices, each of the physical devices including an associated device memory;
and wherein the compute server and the device server are connected through the existing network fabric, the device server directly accesses the system memory of the compute server using the RDMA services of the network fabric, and the compute server directly accesses the system memory of the device server using the RDMA services of the network fabric, the method comprising:
allocating memory space in the system memory of the compute server as memory buffer space on the compute server for data transfers to and from each of the physical devices of the device server, and wherein the device server allocates memory space in the system memory of the device server for data transfers to and from each of the physical devices of the device server; and
logically connecting one or more of the physical devices on the device server to the compute server, including
mapping the device memory associated with each of the physical devices to a copy of the associated device memory in the memory space allocated in the system memory of the compute server for said each physical device, and mapping each of the memory buffer spaces on the compute server to a copy of said each memory buffer space on the device server; and

transferring data from the compute server to one of the physical devices, including
the compute server writing the data inside the memory buffer space of the compute server allocated for said one of the physical devices, and triggering an RDMA transfer of the data to the device server, and

the device server receiving the data, by the triggered RDMA transfer, from said allocated memory buffer space of the compute server.

2. The method according to claim 1 , wherein the triggering an RDMA transfer includes the compute server writing a signal in a memory mapped register on the compute server.

3. The method according to claim 2 , wherein the transferring data from the compute server to the one of the physical devices further includes the compute server synchronizing the data in a shared memory space on the device server.

4. The method according to claim 3 , wherein the device server receiving the data includes the device server accessing the data from the shared memory space on the device server.

5. The method according to claim 4 , wherein the synchronizing the data in a shared memory space on the device server includes synchronizing the data in the shared memory space on the device server before the device server accessing the data from the shared memory space on the device server.

6. The method according to claim 1 , wherein the mapping the device memory associated with each of the physical devices to a copy of the associated device memory in the memory space allocated in the system memory of the compute server for said each physical device includes shadowing and remapping the memories associated with the physical devices to the copies of the associated device memories on the compute server.

7. The method according to claim 6 , wherein the shadowing and remapping the memories associated with the physical devices to the copies of the associated device memories on the compute server includes using a device mapping component on the device server to shadow and remap the memories associated with the physical devices to the copies of the associated device memories on the compute server.

8. The method according to claim 1 , wherein the mapping each of the memory buffer spaces on the compute server to a copy of said each memory buffer space on the device server includes shadowing and remapping the memory buffer spaces on the compute server to the copies thereof on the device server.

9. The method according to claim 8 , wherein the shadowing and remapping the memory buffer spaces on the compute server to copies thereon on the device server includes using a client mapping component on the compute server to shadow and remap the memory buffer spaces on the compute server to the copies thereof on the device server.

10. The method according to claim 1 , wherein the logically connecting one or more of the physical devices on the device server to the compute server further includes re-mapping interrupt vector addresses.

11. A system architecture for attaching remote physical devices through an existing network fabric providing remote direct memory access (RDMA) services, the system architecture comprising:
a compute server including a compute server system memory, and running an operating system and one or more remote device drivers;
a device server including a device server system memory and one or more physical devices, each of the physical devices including an associated device memory; and
wherein the compute server and the device server are connected through the existing network fabric, the device server directly accesses the system memory of the compute server using the RDMA services of the network fabric, and the compute server directly accesses the system memory of the device server using the RDMA services of the network fabric;
a remote interface (RI) component of the compute server to allocate memory space in the system memory of the compute server as memory buffer space on the compute server for data transfers to and from each of the physical devices of the device server, and wherein the device server allocates memory space in the system memory of the device server for data transfers to and from each of the physical devices of the device server; and
a system mapping function to logically connect one or more of the physical devices on the device server to the compute server, including mapping the device memory associated with each of the physical devices to a copy of the associated device memory in the memory space allocated in the system memory of the compute server for said each physical device, and mapping each of the memory buffer spaces on the compute server to a copy of said each memory buffer space on the device server; and wherein
the compute server transfers data to one of the physical devices, including the compute server writing the data inside the memory buffer space of the compute server allocated for said one of the physical devices, and triggering an RDMA transfer of the data to the device server, and the device server receives the data, by the triggered RDMA transfer, from said allocated memory space of the compute server.

12. The system architecture according to claim 11 , wherein triggering an RDMA transfer includes the compute server writing a signal in a memory mapped register on the compute server.

13. The system architecture according to claim 12 , wherein transferring data from the compute server to the one of the physical devices further includes the compute server synchronizing the data in a shared memory space on the device server.

14. The system architecture according to claim 13 , wherein the device server receiving the data includes the device server accessing the data from the shared memory space on the device server.

15. The system architecture according to claim 14 , wherein the synchronizing the data in a shared memory space on the device server includes synchronizing the data in the shared memory space on t

Assignees

• IBM

Inventors

• Franke Hubertus
• Pasetto Davide
• Penner Hartmut

Classifications

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F15/17331 Primary

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title