Using completion queues for RDMA event detection

Patent metadata

Field	Value
Publication number	US-9842083-B2
Application number	US-201514715155-A
Country	US
Kind code	B2
Filing date	May 18, 2015
Priority date	May 18, 2015
Publication date	Dec 12, 2017
Grant date	Dec 12, 2017

Abstract

Systems and methods for using completion queues for Remote Direct Memory Access (RDMA) event detection. An example method may comprise: receiving a request to create a queue pair for processing Remote Direct Memory Access (RDMA) requests using an RDMA-enabled network interface controller (RNIC), the queue pair comprising a send queue and a receive queue; associating the queue pair with a completion queue associated with the RNIC, the completion queue employed to store a plurality of completion queue elements associated with completed work requests; receiving a notification of an interrupt associated with the RNIC; and responsive to determining that at least one of a number of send queues associated with the completion queue or a number of receive queues associated with the completion queue exceeds zero, identifying at least one of: a first application registered to be notified of RDMA send events or a second application registered to be notified of RDMA receive events.

First claim

The invention claimed is:

1. A method, comprising:
receiving, by a processing device, a request to create a queue pair for processing Remote Direct Memory Access (RDMA) requests using an RDMA-enabled network interface controller (RNIC), the queue pair comprising a send queue and a receive queue; associating the queue pair with a completion queue associated with the RNIC, the completion queue employed to store a plurality of completion queue elements associated with completed work requests; receiving a notification of an interrupt associated with the RNIC; responsive to determining that a number of send queues associated with the completion queue exceeds zero, identifying an application registered to be notified of RDMA send events; and awakening the application.

2. The method of claim 1 , wherein the send queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing incoming RDMA data.

3. The method of claim 1 , wherein the receive queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing outgoing RDMA data.

4. The method of claim 1 , wherein associating the queue pair with the completion queue comprises incrementing at least one of: a first counter reflecting the number of send queues associated with the completion queue or a second counter reflecting the number of receive queues associated with the completion queue.

5. The method of claim 4 , further comprising:
responsive to receiving a request to destroy the queue pair, decrementing at least one of: the first counter or the second counter.

6. The method of claim 1 , further comprising:
receiving a request for RDMA send event notifications initiated by the application.

7. A system, comprising:
a memory; and a processing device operatively coupled to the memory, the processing device to: receive a request to create a queue pair for processing Remote Direct Memory Access (RDMA) requests using an RDMA-enabled network interface controller (RNIC), the queue pair comprising a send queue and a receive queue; associate the queue pair with a completion queue associated with the RNIC, the completion queue employed to store a plurality of completion queue elements associated with completed work requests; receive a notification of an interrupt associated with the RNIC; responsive to determining that at least one of a number of send queues associated with the completion queue or a number of receive queues associated with the completion queue exceeds zero, identify at least one of: a first application registered to be notified of RDMA send events or a second application registered to be notified of RDMA receive events; and awaken at least one of: the first application or the second application.

8. The system of claim 7 , wherein the send queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing incoming RDMA data.

9. The system of claim 7 , wherein the receive queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing outgoing RDMA data.

10. The system of claim 7 , wherein to associate the queue pair with the completion queue, the processing device is to increment at least one of: a first counter reflecting the number of send queues associated with the completion queue or a second counter reflecting the number of receive queues associated with the completion queue.

11. The system of claim 10 , wherein the processing device is further to:
responsive to receiving a request to destroy the queue pair, decrement at least one of: the first counter or the second counter.

12. The system of claim 7 , wherein the processing device is further to:
receive at least one of: a first application-initiated request for RDMA send event notifications or a second application-initiated request for RDMA receive event notifications.

13. A computer-readable non-transitory storage medium comprising executable instructions to cause a processing device to: receive, by the processing device, a request to create a queue pair for processing Remote Direct Memory Access (RDMA) requests using an RDMA-enabled network interface controller (RNIC), the queue pair comprising a send queue and a receive queue; associate the queue pair with a completion queue associated with the RNIC, the completion queue employed to store a plurality of completion queue elements associated with completed work requests; receive a notification of an interrupt associated with the RNIC; responsive to determining that a number of receive queues associated with the completion queue exceeds zero, identify an application registered to be notified of RDMA receive events; and awaken the application.

14. The computer-readable non-transitory storage medium of claim 13 , wherein the send queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing incoming RDMA data.

15. The computer-readable non-transitory storage medium of claim 13 , wherein the receive queue is provided by a data structure employed to store a plurality of work queue elements, each work queue element comprising a pointer to a memory buffer for storing outgoing RDMA data.

16. The computer-readable non-transitory storage medium of claim 13 , further comprising executable instructions causing the processing device, responsive to associating the queue pair with the completion queue, to increment at least one of: a first counter reflecting the number of send queues associated with the completion queue or a second counter reflecting the number of receive queues associated with the completion queue.

Assignees

• Red Hat Israel Ltd

Inventors

• Tsirkin Michael

Classifications

• H04L49/3027

  Output queuing · CPC title

• G06F15/167 Primary

  using a common memory, e.g. mailbox · CPC title

• H04L12/66

  Arrangements for connecting between networks having differing types of switching systems, e.g. gateways · CPC title

• H04L49/901

  using storage descriptor, e.g. read or write pointers · CPC title

• H04L12/06

  Answer-back mechanisms or circuits · CPC title