Method and system for flexible credit exchange within high performance fabrics

Patent metadata

Field	Value
Publication number	US-9917787-B2
Application number	US-201615184375-A
Country	US
Kind code	B2
Filing date	Jun 16, 2016
Priority date	Dec 20, 2013
Publication date	Mar 13, 2018
Grant date	Mar 13, 2018

Abstract

Method, apparatus, and systems for implementing flexible credit exchange within high performance fabrics. Available buffer space in a receive buffer on a receive-side of a link is managed and tracked at the transmit-side of the link using credits. Peer link interfaces coupled via a link are provided with receive buffer configuration information that specifies how the receive buffer space in each peer is partitioned and space allocated for each buffer, including a plurality of virtual lane (VL) buffers. Credits are used for tracking buffer space consumption and in credits are returned from the receive-side indicating freed buffer space. The peer link interfaces exchange credit organization information to inform the other peer of how much space each credit represents. In connection with data transfer over the link, the transmit-side de-allocates credits based on an amount of buffer space to be consumed in applicable buffers in the receive buffer. Upon space being freed in the receive buffer, the receive-side returns credit ACKnowledgements (ACKs) identifying a VL for which space has been freed.

First claim

What is claimed is:

1. An apparatus, comprising:
a receive port and a transmit port to:
receive data defining an initial receive buffer configuration of a receive port of a peer apparatus to which the apparatus is configured to be linked in communication, the receive buffer configuration comprising an overall buffer space partitioned into a plurality of buffers including a plurality of virtual lane (VL) buffers, each allocated for a respective VL, the data including an initial allocation of credits for the VL buffers;
receive credit organization information conveying an amount of buffer space represented by each credit, the credit organization information including an allocation unit (AU) corresponding to a minimum block size used for buffer allocation in the receive buffer, each buffer credit corresponding to a buffer space that is a multiple of the AU;
receive credit organization information from the peer apparatus from which a size of the AU can be determined and from which a credit return ACKnowledgement (ACK) unit (CU) defining a number of receiver AU represented by each credit returned from the receiver via credit return ACKs can be determined;
transmit data outbound onto the link to be received by the receive port of the peer apparatus;
de-allocate buffer credits at the transmit port for applicable buffers in the receive buffer based on an amount of transmitted data and the credit organization information;
receive credit return ACKs from the peer apparatus, each credit return ACK identifying a VL and one or more credits conveying an amount of freed buffer space for a buffer allocated for the VL and based on the credit organization information; and
track, at the transmit port, available credits for each buffer in the receive buffer based on the de-allocated buffer credits and credits returned via the credit return ACKs using the AU and CU.

2. The apparatus of claim 1 , wherein the credit organization information received from the peer apparatus comprises an AU variable (vAU) and a CU variable (vCU), each of which is encoded to define a corresponding AU and CU using a logarithmic scale.

3. The apparatus of claim 2 , wherein the peer apparatus includes a peer transmit port and a peer receive port, and wherein the apparatus further comprises:
a receive buffer, having buffer space partitioned into a plurality of buffers; and
the apparatus further to,
send first credit organization information for the receive buffer of the apparatus to the peer apparatus, the first credit organization information including a first vAU and a first vCU;
receive second credit organization information for the receive buffer of the peer apparatus at the receive port of the apparatus, the second credit organization information including a second vAU and a second vCU;
employ a first AU and a first CU corresponding to the first vAU and first vCU for determining return credit ACKs to be transmitted to the link peer interface in response to buffer space being freed in the receive buffer in the receive port of the apparatus; and
employ a second AU and second CU corresponding to the second vAU and second vCU to manage and track credits for the receive buffer in the peer apparatus.

4. The apparatus of claim 1 , wherein a credit return ACK comprises a multi-bit field that is encoded using a logarithmic scale.

5. The apparatus of claim 1 , wherein data is transferred over the link via link transfer packets, and credit return ACKs are encoded in a sideband field of a link transfer packet, and wherein the apparatus to extract credit return ACK data from the sideband field of a link transfer packet.

6. The apparatus of claim 1 , wherein data is transferred over the link via data units, and wherein multiple credit return ACKs corresponding to multiple VLs are encoded in a single data unit, and wherein the apparatus to extract credit return ACK data for the multiple VLs from the single data unit.

7. The apparatus of claim 1 , further to:
receive buffer reconfiguration information defining a reconfiguration of buffer space to be implemented for the receive buffer; and
based on the buffer reconfiguration information, adjust available credits tracked at the transmit port for respective buffers in the receive buffer for which space has been reconfigured.

8. The apparatus of claim 1 , wherein the receive buffer is partitioned into a plurality of VL buffers, each dedicated to a respective VL, and at least one buffer that is shared among multiple VLs.

9. The apparatus of claim 8 , wherein the plurality of buffers partitioned from the receive buffer space includes a global buffer that is shared among all of the VLs.

10. The apparatus of claim 8 , wherein the apparatus and the peer apparatus are configured to be implemented in neighbor fabric devices in a fabric comprising a plurality of fabric devices coupled via a plurality of fabric links, and wherein data is transmitted across the fabric via data units comprising flits divided from corresponding fabric packets, and the apparatus further to:
determine whether there is adequate receive buffer space for buffering an entirety of the flits for a fabric packet, wherein the flits for the fabric packet are to be received over a single VL, and the adequate buffer space includes available buffer space in the VL buffer allocated for the VL and any shared buffer space allocated to the VL.

11. The apparatus of claim 8 , further to:
track available credits for each VL and a shared buffer pool that is shared amongst a plurality of VLs
when buffer space is available for a VL buffer associated with a VL used for transmission of data, de-allocate credits for that VL buffer; otherwise
when buffer space is not available for the VL buffer associated with the VL used for transmission of data, de-allocate credits for the shared buffer pool.

12. The apparatus of claim 11 , further to:
in response to receiving credit return ACKs identifying a VL that is amongst the VLs sharing the shared buffer pool, return credits to the shared buffer pool while the buffer space represented by the tracked credits for the shared buffer pool is less than the size of the shared buffer pool; otherwise
return credits to the VLs identified in the credit return ACKs.

13. The apparatus of claim 1 , wherein the transmit port includes a credit manager that tracks available buffer space in the receive buffer of the peer apparatus.

14. The apparatus of claim 1 , wherein the apparatus further comprises a memory and at least one processor core, and wherein the at least one processor core and the memory are packaged within the same chip package.

15. The apparatus of claim 1 , wherein the apparatus further comprises a memory and at least one processor core, and wherein the at least one processor core and the memory are integrated on a System on a Chip.

16. A method, comprising:
receiving from a link peer an initial allocation of credits for each of a plurality of buffers implemented in a receive buffer of a receiver in the link peer, the receive buffer having an overall buffer space partitioned into a plurality of buffers including a plurality of virtual lane (VL) buffers, each allocated for a respective VL;
receiving from the link peer credit organization information conveying an amount of buffer space represented by each credit to be returned from the receiver;
transmitting data from a transmitter outbound onto a link to be received by the receiver;
de-allocating buffer credits for applicable buffers in the receive buffer based on an amount of buffer space in those buffers to be consumed by the transmitted data an

Assignees

• Intel Corp

Inventors

• Rimmer Todd
• Lovett Thomas D
• Cheng Albert

Classifications

• H04L47/2433

  Allocation of priorities to traffic types · CPC title

• H04L49/103

  using a shared central buffer; using a shared memory · CPC title

• H04L49/109

  Integrated on microchip, e.g. switch-on-chip · CPC title

• H04L49/9057

  Arrangements for supporting packet reassembly or resequencing · CPC title

• G06F13/1673

  using buffers · CPC title