Accelerating request/response protocols

Patent metadata

Field	Value
Publication number	US-12093571-B1
Application number	US-202217937167-A
Country	US
Kind code	B1
Filing date	Sep 30, 2022
Priority date	Sep 30, 2022
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

A network device can be used on a client side and a server side to accelerate request/response protocols for object transfers between the client side and the server side. The network device can interface with an application executing on the host device, which can allow the network device to expose different operations that can enable acceleration of different higher level object transfer commands. The network device can expose operations to pre-post anonymous receive buffers and client buffers even before the server side responds to the object transfer commands, which can allow the client buffers to be available to the server early on. Small transfers can be performed without remote direct memory access (RDMA) and can be received in the anonymous receive buffers at the server side. Bulk object data can be sent via RDMA for direct data placement into the client buffers to enable zero copy.

First claim

What is claimed is:

1. A method performed by a client network device, comprising:
receiving, via a submission queue (SQ), a create command slot request from a client host application executing on a client host device to create a command slot to perform an object transfer with a server host device;
creating the command slot by allocating resources to perform the object transfer, wherein allocating the resources includes establishing a transport protocol connection with a server network device of the server host device;
receiving, via the SQ, a first unsolicited message to perform the object transfer based on a first application protocol command to get object data from the server host device;
configuring the command slot to perform cryptographic operations on the object data;
receiving, via the SQ, a first request to get a first portion of the object data from the server host device, the first request comprising address information associated with a client buffer for placing the first portion of the object data, an object offset, and a first sequence identifier (ID);
assigning a first tag corresponding to the address information associated with the client buffer for direct data placement (DDP) of the first portion of the object data to be transferred from the server host device using remote direct memory access (RDMA);
sending the address information associated with the client buffer, the object offset, the first sequence ID, and the first tag to the server network device using the transport protocol connection, wherein the server network device forwards a client request ID, the object offset, and the first sequence ID to a server host application executing on the server host device, and wherein the server host application sends a message to the server network device to send the first portion of the object data to the client network device based on the object offset and the client request ID, the message comprising a local memory address of the server host device where the object data is stored;
receiving the first portion of the object data via a plurality of RDMA writes performed by the server network device based on the message received from the server host application; and
performing the DDP of the received first portion of the object data into the client buffer using the first tag.

2. The method of claim 1 , further comprising:
receiving, via the SQ, a second unsolicited message to perform the object transfer based on a second application protocol command to send the object data to the server host device;
receiving, via the SQ, a second request to send a second portion of the object data from the client buffer to the server host device, the second request comprising address information associated with the client buffer where the second portion of the object data is stored, a size of the second portion, and a second sequence ID;
sending the address information associated with the client buffer, the size of the second portion, and the second sequence ID to the server network device using the transport protocol connection, wherein the server network device sends a client request ID, the size of the second portion, and the second sequence ID to the server host application, wherein the server host application allocates local memory in the server host device for placing the object data;
receiving an RDMA read request from the server network device to perform RDMA reads from the client buffer, wherein the RDMA read request includes a second tag assigned by the server network device for the DDP of the second portion of the object data to be transferred from the client buffer to the local memory in the server host device using the RDMA reads, and address information associated with the local memory in the server host device for placing the second portion of the object data received via the RDMA reads;
receiving the second portion of the object data from the client buffer in response to the RDMA read request; and
sending the second portion of the object data to the server host device for storing in the local memory.

3. The method of claim 1 , wherein the first portion of the object data is received as multiple packets based on the transport protocol connection.

4. The method of claim 2 , wherein contents of the first unsolicited message and the second unsolicited message are stored in receive (RX) buffers in the server host device that are pre-posted by the client host application in an RX queue (RQ) in the server network device.

5. A method performed by a client network device, comprising:
receiving a message to perform an object transfer from a server network device based on an application protocol command;
receiving a request to start the object transfer from the server network device, the request comprising information associated with a client buffer for placing object data received from the server network device;
assigning, based on the information associated with the client buffer, a tag for direct data placement (DDP) of the object data to be transferred from the server network device using remote direct memory access (RDMA);
sending the information associated with the client buffer and the tag for the DDP to the server network device;
receiving the object data in the client buffer via RDMA writes from the server network device based on the tag; and
sending a completion message to a client host application indicating that the object transfer has completed when a client buffer scoreboard that accumulates completion of the RDMA writes is full.

6. The method of claim 5 , wherein the server network device sends a client request identifier (ID) and a requested data range for the object data to a server host application of a server based on the request.

7. The method of claim 6 , wherein the server network device receives a message from the server host application to send the object data to the client network device based on the requested data range of the object data and the client request ID, the message from the server host application comprising location information associated with a local memory of the server where the object data is stored.

8. The method of claim 7 , wherein the information associated with the client buffer includes a client buffer address, and the client request ID corresponds to the client buffer address, and wherein the requested data range corresponds to an offset in the local memory of the server where the object data is stored.

9. The method of claim 5 , further comprising:
prior to receiving the message to perform the object transfer from the server network device, receiving a request to create a command slot to be used for performing the object transfer from the server network device;
creating the command slot by allocating a command slot identifier; and
configuring the command slot to perform cryptographic operations on the object data.

10. The method of claim 5 , wherein the application protocol command is an object GET command.

11. The method of claim 5 , further comprising:
receiving another message to perform the object transfer to the server network device based on another application protocol command;
receiving another request to send the object data to the server network device, the other request comprising information associated with the client buffer where the object data is stored and a size of the object data;
sending the information associated with the client buffer and the size of the object data to the server network device, wherein the server network device sends a second client request ID and the size of the object data to a server host application of a server, and wherein the server host application allocates local memory in the se

Assignees

• Amazon Tech Inc

Inventors

• Schmeilin Evgeny
• Shalev Leah
• Fodi Ghanim
• Agroskin Shay
• Broner Gilad

Classifications

• G06F3/0659 Primary

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/0668 Primary

  adopting a particular infrastructure · CPC title