Sending packets using optimized PIO write sequences without sfences and out of order credit returns

Patent metadata

Field	Value
Publication number	US-9785359-B2
Application number	US-201615054325-A
Country	US
Kind code	B2
Filing date	Feb 26, 2016
Priority date	Feb 26, 2016
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Methods and apparatus for sending packets using optimized PIO write sequences without sfences and out-of-order credit returns. Sequences of Programmed Input/Output (PIO) write instructions to write packet data to a PIO send memory are received by a processor in an original order and executed out of order, resulting in the packet data being written to send blocks in the PIO send memory out of order, while the packets themselves are stored in sequential order once all of the packet data is written. The packets are egressed out of order by egressing packet data contained in the send blocks to an egress block using a non-sequential packet order that is different than the sequential packet order. In conjunction with egressing the packets, corresponding credits are returned in the non-sequential packet order. A block list comprising a linked list and a free list are used to facilitate out-of-order packet egress and corresponding out-of-order credit returns.

First claim

The invention claimed is:

1. A method comprising:
receiving sequences of Programmed Input/Output (PIO) write instructions to write packet data for respective packets stored in memory on a host to a PIO send memory on a network adaptor;
writing the packet data into the PIO send memory without using sfences, the packet data being written to send blocks in the PIO send memory such that the packet data is stored in a sequential packet order; and
forwarding packet data stored in associated send blocks in the PIO send memory for egress to a network via the network adaptor, wherein the packet data is forwarded for egress out of order by using a non-sequential packet order that is different than the sequential packet order; and
returning credits to the host in conjunction with packet data stored in associated send blocks being forwarded for egress, wherein the credits are returned in the non-sequential packet order.

2. The method of claim 1 , further comprising:
implementing a free list containing a list of send blocks in the PIO send memory that are free to write to;
implementing a block list comprising a linked list of send blocks containing packet data that is linked in a manner that tracks the non-sequential packet order; and
updating the free list and the block list in conjunction with the packet data stored in the associated send blocks being forwarded for egress.

3. The method of claim 2 , further comprising:
partitioning the PIO send memory into a plurality of send contexts, each send context organized as a sequence of send blocks; and
implementing a respective pair of free list and block list for each of the plurality of send contexts.

4. The method of claim 3 , further comprising:
storing packet data for a packet in a set of one or more send blocks in a send context of the PIO send memory;
reading the packet data from the send context into an egress FIFO (First-in, First-out) buffer;
generating a credit return corresponding to a number of blocks in the set of one or more blocks that have been read out for egress; and
updating the free list to reflect that the set of one or more blocks in the PIO send memory are free.

5. The method of claim 4 , further comprising incrementing a free list tail pointer in the free list by the number of blocks in the set of one or more blocks that have been read out for egress.

6. The method of claim 4 , wherein the packet data for the packet is read out of order and the method further remaps values in the free list to point to out-of-order locations in the send context corresponding to the locations of the set of one or more send blocks in the send context.

7. The method of claim 4 , further comprising:
updating a block list end pointer in the block list to point to a location in the block list containing the location in the PIO send memory of the last block in the set of one or more blocks.

8. The method of claim 2 , further comprising:
launching a packet for egress, the packet including packet data stored in the PIO send memory in one or more send blocks including a first send block and a last send block; and
updating a free list head pointer in the free list to point to a next send block in the PIO send memory following the last send block.

9. The method of claim 2 , further comprising:
determining, via the free list, whether one or more send blocks are available to write in the PIO send memory; and
writing packet data into the one or more blocks if the free list indicates the one or send blocks are free, otherwise waiting to write the packet data into the one or more send blocks until the free list indicates the one or more send blocks are free.

10. The method of claim 2 , wherein the packet data for a given packet is forwarded from the PIO send memory to be egressed by forwarding packet data contained in one or more send blocks, the method further comprising updating the block list to reflect an order in which the packet data in the send blocks is forwarded for egress.

11. A method comprising:
partitioning memory space in a Programmed Input/Output (PIO) send memory into a plurality of send contexts, each comprising a memory buffer including a plurality of send blocks configured to store packet data;
implementing a storage scheme using First-in, First-out (FIFO) semantics for each send context under which each send block occupies a respective FIFO slot in a FIFO buffer having a FIFO order and data for a given packet is stored in one or more send blocks occupying one or more respective sequential FIFO slots in a FIFO order;
receiving packet data written to send blocks out of order such that for at least a portion of packets send blocks are filled with packet data in a different order than the FIFO order, the packet data being written to the send blocks such that the packet data is stored in a send context containing the packet data in a sequential packet order;
egressing a plurality of packets out of order by egressing packet data contained in send blocks to an egress block, wherein the packets are egressed using a non-sequential packet order that is different than the sequential packet order; and
returning credits in conjunction with egressing the plurality of packets out of order, wherein the credits are returned in the non-sequential packet order.

12. The method of claim 11 , further comprising:
for each send context,
implementing a free list containing a list of send blocks in the send context that are free to write to;
implementing a block list comprising a linked list of send blocks containing packet data that is linked in a manner that tracks the non-sequential packet order; and
updating the free list and the block list in conjunction with egressing the plurality of packets.

13. The method of claim 12 , further comprising:
storing packet data for a packet in a set of one or more send blocks in a send context of the PIO send memory;
reading the packet data from the send context into an egress FIFO (First-in, First-out) buffer;
generating a credit return corresponding to a number of blocks in the set of one or more blocks that have been read out for egress; and
updating the free list to reflect that the set of one or more blocks in the PIO send memory are free.

14. An apparatus, comprising:
an input/output (IO) interface, configured to be coupled to a host;
a transmit engine coupled to the IO interface and including,
a Programmed Input/Output (PIO) send memory;
an egress block, operatively coupled to the PIO send memory; and
circuitry and logic to,
partition the PIO send memory into a plurality of send contexts, each comprising a plurality of sequential send blocks;
implement a storage scheme using First-in, First-out (FIFO) semantics for each send context under which each send block occupies a respective FIFO slot in a FIFO buffer having a FIFO order and data for a given packet is stored in one or more send blocks occupying one or more respective sequential FIFO slots in a FIFO order;
receive packet data for a plurality of packets and store the packet data in a plurality of send blocks in a send context, wherein the packet data for respective packets are stored in sequential sets of one or more send blocks comprising a sequential packet order;
egress packets from the send context to the egress block as blocks of packet data, wherein at least a portion of the packets are egressed to the egress block out-of-order in a non-sequential packet order; and
return credits via the IO interface in conjunction with the packets being egressed to the egress block, wherein the credits are returned in the non-sequential packet order.

Assignees

• Intel Corp

Inventors

• Mutha Yatin
• Debbage Mark

Classifications

• H04L49/901

  using storage descriptor, e.g. read or write pointers · CPC title

• G06F3/0644

  Management of space entities, e.g. partitions, extents, pools · CPC title

• G06F2213/0026

  PCI express · CPC title

• G06F3/061 Primary

  Improving I/O performance · CPC title

• G06F13/4068

  Electrical coupling · CPC title