Method, apparatus, and system for QoS within high performance fabrics

Patent metadata

Field	Value
Publication number	US-9648148-B2
Application number	US-201314139930-A
Country	US
Kind code	B2
Filing date	Dec 24, 2013
Priority date	Dec 24, 2013
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

Method, apparatus, and systems for implementing Quality of Service (QoS) within high performance fabrics. A multi-level QoS scheme is implemented including virtual fabrics, Traffic Classes, Service Levels (SLs), Service Channels (SCs) and Virtual Lanes (VLs). SLs are implemented for Layer 4 (Transport Layer) end-to-end transfer of fabric packets, while SCs are used to differentiate fabric packets at the Link Layer. Fabric packets are divided into flits, with fabric packet data transmitted via fabric links as flits streams. Fabric switch input ports and device receive ports detect SC IDs for received fabric packets and implement SC-to-VL mappings to determine VL buffers to buffer fabric packet flits in. An SL may have multiple SCs, and SC-to-SC mapping may be implemented to change the SC for a fabric packet as it is forwarded through the fabric, while maintaining its SL. A Traffic Class may include multiple SLs, enabling request and response traffic for an application to employ separate SLs.

First claim

What is claimed is:

1. A method comprising:
implementing a plurality of Service Levels (SLs) for transferring fabric packets between fabric endpoints in a fabric comprising a plurality of fabric links coupled between fabric ports of a plurality of fabric devices;
assigning one or more Service Channels (SCs) to each Service Level;
at each of at least a portion of the fabric ports, assigning the SCs to Virtual Lanes (VLs); and
dividing each fabric packet into a plurality of flits, and effecting transfer of fabric packets between fabric endpoints by transferring the flits for the fabric packets over the fabric links using the SCs;
wherein the VL used to buffer the flits for a given fabric packet to be transmitted across a given fabric link is determined based on an SC implemented for transferring the flits across the fabric link.

2. The method of claim 1 , wherein the Service Levels are implemented in a Transport Layer; and the Service Channels are implemented at a Link Layer for the fabric.

3. The method of claim 1 , further comprising implementing a plurality of Traffic Classes, wherein each of the Traffic Classes is assigned one or more of the Service Levels, and each of the Traffic Classes is allocated to a given Transport Layer or application that manages data transfers at a Transport Layer.

4. The method of claim 3 , further comprising implementing two of the SLs for one of the Traffic Classes under which request traffic for the one of the Traffic Classes is assigned a first of the SLs and response traffic for the one of the Traffic Classes is assigned a second of the SLs.

5. The method of claim 4 , further comprising implementing a plurality of virtual fabrics (vFabrics), wherein each of the vFabrics is associated with one of the Traffic Classes and at least one of the vFabrics is implemented over a subset of the fabric ports in the fabric.

6. The method of claim 1 , wherein the fabric packet includes an SC identifier (ID) and the SL implemented for transferring the fabric packet across the fabric is implied by the fabric packet's SC ID.

7. The method of claim 1 , further comprising:
assigning a first of the SCs to a fabric packet at a first fabric endpoint, the fabric packet to be transferred through the fabric along a forwarding path to a second fabric endpoint, the forwarding path comprising a plurality of fabric links and including traversal through at least one fabric switch;
employing the first of the SCs for forwarding the flits for the fabric packet along a first portion of the forwarding path including at least one fabric link;
at the fabric switch, changing the first of the SCs for the fabric packet from the first of the SCs to a second of the SCs; and
employing the second of the SCs for forwarding the flits for the fabric packet along a second portion of the forwarding path including at least one fabric link.

8. The method of claim 1 , wherein the fabric comprises a heterogeneous fabric including fabric devices supporting different numbers of VLs.

9. The method of claim 8 , wherein the flits are transmitted from a transmitting fabric port implementing a first number of the VLs to a receiving fabric port of a link peer implementing a second number of the VLs that is less than the first number of the VLs, the method further comprising:
performing VL contraction mapping, under which the VLs in the transmitting fabric port are mapped down to the VLs in the receiving fabric port.

10. The method of claim 8 , wherein the flits are transmitted from a transmitting fabric port implementing a first number of the VLs to a receiving fabric port of a link peer implementing a second number of the VLs that is greater than the first number of the VLs, the method further comprising:
performing VL expansion mapping, under which the VLs in the transmitting fabric port are mapped up to the VLs in the receiving fabric port.

11. The method of claim 1 , further comprising implementing traffic prioritization, wherein the SLs are assigned priority levels, and transmission of the flits for fabric packets associated with an SL having a higher priority level may preempt transmission of the flits for fabric packets associated with an SL having a relatively lower priority.

12. The method of claim 1 , further comprising:
assigning a first SC to a fabric packet at a source fabric endpoint, the fabric packet to be transferred through the fabric along a forwarding path comprising a plurality of fabric links to a destination fabric endpoint;
storing an SC identifier (ID) for the first SC in a fabric packet header;
dividing the fabric packet into a plurality of flits, including a head flit, a plurality of body flits, and a tail flit, wherein data in the head flit includes the SC ID;
receiving the flits at a fabric switch, and detecting the head flits;
inspecting the head flit of the fabric packet to determine its SC ID; and
employing an SC-to-VL map to determine which VL is selected for transmission of the flits for the fabric packet from the fabric switch to a next hop along the forwarding path.

13. The method of claim 12 , further comprising:
employing an SC-to-SC map to change the SC from the first SC to a second SC; and
replacing the SC ID in the head flit with the SC ID for the second SC; and
employing the current SC for the fabric packet to determine a VL buffer to use to transfer the flits for the fabric packet across the fabric links; and
wherein, for at least the one fabric packet, the flits for the fabric packet are forwarded along the forwarding path using at least two different virtual lanes VLs.

14. An apparatus comprising:
memory comprising a transmit buffer;
a fabric transmit port; and
circuitry and logic to:
partition the transmit buffer into a plurality of transmit Virtual Lane (VL) buffers and associate each of the transmit VL buffers with a respective VL used by the transmit port;
implement a Service Channel (SC)-to-transmit VL map, wherein input SC identifiers (IDs) are mapped to the VLs used by the transmit port;
receive fabric packet data corresponding to a fabric packet to be transmitted out the fabric transmit port onto a fabric link;
retrieve an SC ID from the received fabric packet data;
use the SC ID as an input to the SC-to-transmit VL map to identify a transmit VL buffer for transmitting the fabric packet data via the fabric transport port; and
buffer the fabric packet data in the transmit VL buffer that is identified.

15. The apparatus of claim 14 , wherein the apparatus is configured to effect Quality of Service (QoS) for transmission of fabric packets in the fabric, the QoS including multiple Service Levels (SLs), wherein at least some of the SLs are prioritized and an SC has an implied SL, and the apparatus further comprises:
circuitry and logic to arbitrate among the VLs and select VLs from which the fabric packet data is pulled to be transmitted from the transmit port, under which the VLs having higher priority are enabled to preempt the VLs having lower priority.

16. The apparatus of claim 14 , further comprising:
memory comprising a receive buffer;
a receive port; and circuitry and logic to:
partition the receive buffer into a plurality of receive Virtual Lane (VL) buffers and associate each of the receive VL buffers with a respective receive VL;
implement a Service Channel (SC)-to-receive VL map, wherein input SC identifiers (IDs) are mapped to the receive VL buffers;
receive fabric packet data corresponding to a fabric packet at the receive port;
retrieve an SC ID from the received fabric packet data;
using the SC ID as an input

Assignees

• Intel Corp

Inventors

• Rimmer Todd
• Lovett Thomas D
• Cheng Albert

Classifications

• H04L49/358

  Infiniband Switches · CPC title

• H04L69/22 Primary

  Parsing or analysis of headers · CPC title

• H04L49/70

  Virtual switches · CPC title

• H04L47/2491

  Mapping quality of service [QoS] requirements between different networks · CPC title

• H04L49/30

  Peripheral units, e.g. input or output ports · CPC title