Systems and methods for allocating bandwidth across a cluster of accelerators

Patent metadata

Field	Value
Publication number	US-10848440-B2
Application number	US-201916359700-A
Country	US
Kind code	B2
Filing date	Mar 20, 2019
Priority date	Mar 20, 2019
Publication date	Nov 24, 2020
Grant date	Nov 24, 2020

Abstract

The present disclosure provides methods and systems directed to providing quality of service to cluster of accelerators. The system can include a root connector; an interconnect switch communicatively coupled to the root connector over a plurality of lanes comprising a first set of lanes and a second set of lanes, wherein the first set of lanes are associated with a first virtual communication channel and a second set of lanes are associated with a second virtual communication channel; a first accelerator communicatively coupled to the interconnect switch and associated with a first traffic class identifier corresponding to first communication traffic communicated over the first set of lanes; and a plurality of accelerators communicatively coupled to the interconnect switch and associated with a second traffic class identifier that corresponds to second communication traffic having lower priority than the first communication traffic and that is communicated over the second set of lanes.

First claim

What is claimed is:

1. A computer system, comprising:
a root connector, configured to connect to a memory;
an interconnect switch communicatively coupled to the root connector over a plurality of lanes comprising a first set of lanes and a second set of lanes, wherein the first set of lanes are associated with a first virtual communication channel and a second set of lanes are associated with a second virtual communication channel;
a first accelerator communicatively coupled to the interconnect switch and associated with a first traffic class identifier corresponding to first communication traffic communicated over the first set of lanes; and
a plurality of accelerators communicatively coupled to the interconnect switch and associated with a second traffic class identifier that corresponds to second communication traffic having lower priority than the first communication traffic and that is communicated over the second set of lanes, wherein the first communication traffic and the second communication traffic comprising processing commands and memory access requests.

2. The system according to claim 1 , wherein the first set of lanes includes more lanes than the second set of lanes.

3. The system according to claim 1 , wherein the first communication traffic further comprises processing commands related to image recognition and the second communication traffic further comprising processing commands related to voice recognition.

4. The system according to claim 1 , wherein the interconnect switch is a peripheral component interconnect express (PCIe) switch, the root connector is a PCIe root complex, and the traffic class identifier is in a transaction layer packet (TLP).

5. The system according to claim 1 , wherein the interconnect switch is a mesh network.

6. The system according to claim 1 , further comprising one or more other accelerators communicatively coupled to the interconnect switch and associated with the first traffic class identifier.

7. The system according to claim 1 , wherein the root connector includes a mapping configuration associating the first traffic class identifier with the first virtual communication channel and the second traffic class identifier with the second virtual communication channel.

8. A computer system, comprising:
a root connector, configured to connect to a memory;
an interconnect switch communicatively coupled to the root connector over a plurality of lanes comprising a first set of lanes and a second set of lanes, wherein the first set of lanes are associated with the first virtual communication channel and a second set of lanes are associated with a second virtual communication channel;
a first accelerator communicatively coupled to the interconnect switch and associated with a first traffic class identifier corresponding to first communication traffic communicated over the first set of lanes;
a plurality of accelerators communicatively coupled to the interconnect switch and associated with a second traffic class identifier that corresponds to second communication traffic having lower priority than the first communication traffic and that is communicated over the second set of lanes; and
a processor communicatively coupled to the root connector and configured to generate processing commands associated with the first communication traffic and the second communication traffic.

9. A method comprising:
establishing a first virtual communication channel over a first set of lanes between an interconnect switch and a root connector;
establishing a second virtual communication channel over a second set of lanes between the interconnect switch and the root connector;
acquiring first communication traffic associated with a first traffic class identifier over the first virtual communication channel;
acquiring second communication traffic associated with a second traffic class identifier over the second virtual communication channel, wherein the second communication traffic have lower priority than the first communication traffic, the first communication traffic and the second communication traffic are based on processing commands, and the processing commands include higher priority processing commands assigned with the first traffic class identifier and lower priority commands assigned with the second traffic class identifier;
routing the first communication traffic to a first accelerator associated with the first traffic class identifier; and
routing the second communication traffic to one or more other accelerators associated with the second traffic class identifier.

10. The method according to claim 9 , wherein the first communication traffic and the second communication traffic are based on processing commands generated according to artificial intelligence applications.

11. The method according to claim 9 , further comprising establishing a virtual buffer for each virtual communication channel.

12. The method according to claim 9 , further comprising:
prior to establishing the first virtual communication channel and the second virtual communication channel, allocating lanes between the interconnect switch and the root connector to virtual communication channels, wherein the allocated lanes include the first set of lanes allocated to the first virtual communication channel and the second set of lanes allocated to the second virtual communication channel.

13. The method according to claim 9 , wherein the first traffic class identifier is mapped to the first virtual communication channel and the second traffic class identifier is mapped to the second virtual communication channel.

14. The method according to claim 9 , further comprising associating each accelerator of a plurality of accelerators communicating with the interconnect switch with a traffic class identifier.

15. The method according to claim 9 , further comprising mapping memory access requests of the first accelerator associated with the first traffic class identifier to a first virtual communication channel and mapping memory access requests of an accelerator of the one or more accelerators associated with the second traffic class identifier to a second virtual communication channel.

16. A method comprising:
assigning a plurality of accelerators to a first virtual communication channel of an interconnect switch;
determining if the interconnect switch has met a predetermined bandwidth threshold;
evaluating a workload traffic of a first accelerator of the plurality of accelerators in response to a determination that the interconnect switch has reached the predetermined bandwidth threshold;
assigning the first accelerator a second virtual communication channel in response to the evaluation, wherein the second virtual communication channel transports traffic having a higher priority than traffic transported over the first virtual communication channel;
determining whether the interconnect switch has achieved a better performance based on the assignment of the first accelerator to the second virtual communication channel; and
configuring the interconnect switch based on the determination.

17. The method according to claim 16 , wherein the threshold is based on a percentage of all traffic through the interconnect switch.

18. The method according to claim 16 , wherein configuring the interconnect switch based on the determination further comprises:
assigning the first accelerator to the first virtual communication channel in response to the determination that the interconnect switch does not have a better performance after the assignment of the first a

Assignees

• Alibaba Group Holding Ltd

Inventors

• Zhao Li
• Jin Lingling

Classifications

• H04L49/25 Primary

  Routing or path finding in a switch fabric · CPC title

• H04L49/15 Primary

  Interconnection of switching modules · CPC title

• H04L49/354

  for supporting virtual local area networks [VLAN] · CPC title

• H04L12/66

  Arrangements for connecting between networks having differing types of switching systems, e.g. gateways · CPC title