Peer-to-peer device arrangements in communication fabrics

Patent metadata

Field	Value
Publication number	US-11314677-B2
Application number	US-202117162922-A
Country	US
Kind code	B2
Filing date	Jan 29, 2021
Priority date	May 8, 2017
Publication date	Apr 26, 2022
Grant date	Apr 26, 2022

Abstract

Disaggregated computing architectures, platforms, and systems are provided herein. In one example, a method includes receiving user input to establish a compute unit comprising a host processor and at least two graphics processing units (GPUs) having a peer-to-peer capability. The method also includes instructing a management element for a communication fabric to form the compute unit and communicatively couple the host processor and the at least two GPUs over the communication fabric. The method also includes instructing the management element to establish an isolation function to form the peer arrangement between the at least two GPUs in the communication fabric, where the isolation function isolates a first address domain associated with the at least two GPUs from at least a second address domain associated with the host processor by at least establishing synthetic devices representing the at least two GPUs in the second address domain.

First claim

What is claimed is:

1. A method comprising:
communicatively coupling Peripheral Component Interconnect Express (PCIe) devices over a PCIe fabric; and
establishing a peer-to-peer arrangement between the PCIe devices over the PCIe fabric by at least providing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the PCIe devices from at least a local PCIe address domain associated with a host processor that initiates the peer-to-peer arrangement between the PCIe devices;
wherein the isolation function comprises isolating the device PCIe address domain from the local PCIe address domain by at least establishing synthetic PCIe devices representing the PCIe devices in the local PCIe address domain.

2. The method of claim 1 , further comprising:
redirecting traffic transferred by the host processor for the PCIe devices in the local PCIe address domain for delivery to corresponding ones of the PCIe devices in the device PCIe address domain.

3. The method of claim 1 , further comprising:
adding a selected PCIe device into the peer-to-peer arrangement by at least instantiating the selected PCIe device into the device PCIe address domain and providing a further synthetic device representing the selected PCIe device on the local PCIe address domain; and
removing the selected PCIe device from the peer-to-peer arrangement by at least inactivating the further synthetic device.

4. The method of claim 1 , wherein the PCIe devices comprise PCIe endpoint devices.

5. The method of claim 1 , further comprising:
establishing the isolation function in the PCIe fabric by at least providing address traps that monitor for traffic over the PCIe fabric directed to PCIe addresses associated with the PCIe devices in the local PCIe address domain and translate the PCIe addresses into corresponding PCIe addresses for the PCIe devices in the device PCIe address domain.

6. The method of claim 1 , further comprising:
responsive to peer-to-peer traffic from a first of the PCIe devices indicating the second of the PCIe devices as a destination in the local PCIe address domain, employing a peer-to-peer address trap in the isolation function to receive the peer-to-peer traffic and transfer the peer-to-peer traffic to the second of the PCIe devices in the device PCIe address domain.

7. The method of claim 6 , wherein the peer-to-peer arrangement between the PCIe devices comprises a direct memory access (DMA) link established between memory associated with each of the PCIe devices.

8. The method of claim 1 , further comprising:
in a control processor, establishing the isolation function in PCIe switch circuitry that communicatively couples the PCIe devices and the host processor over the PCIe fabric.

9. The method of claim 1 , wherein the isolation function comprises one or more address traps and one or more address translation tables.

10. The method of claim 1 , further comprising:
selecting PCIe addresses in the isolation function for the PCIe devices in the device PCIe address domain that do not overlap with PCIe address usages of further PCIe devices communicatively coupled on the PCIe fabric.

11. A system, comprising:
a Peripheral Component Interconnect Express (PCIe) fabric configured to communicatively couple PCIe devices with at least a host processor; and
a control processor configured to facilitate a peer-to-peer arrangement between the PCIe devices over the PCIe fabric by at least establishing an isolation function in the PCIe fabric configured to isolate a device PCIe address domain associated with the PCIe devices from at least a local PCIe address domain associated with the host processor that initiates the peer-to-peer arrangement between the PCIe devices by at least establishing synthetic PCIe devices representing the PCIe devices in the local PCIe address domain.

12. The system of claim 11 , comprising:
the isolation function configured to redirect traffic transferred by the host processor for the PCIe devices in the local PCIe address domain for delivery to corresponding ones of the PCIe devices in the device PCIe address domain.

13. The system of claim 11 , comprising:
the control processor configured to add a selected PCIe device into the peer-to-peer arrangement by at least instantiating the selected PCIe device into the device PCIe address domain and providing a further synthetic device representing the selected PCIe device on the local PCIe address domain; and
the control processor configured to remove the selected PCIe device from the peer-to-peer arrangement by at least inactivating the further synthetic device.

14. The system of claim 11 , comprising:
the control processor configured to establish the isolation function in the PCIe fabric by at least providing address traps that monitor for traffic over the PCIe fabric directed to PCIe addresses associated with the PCIe devices in the local PCIe address domain and translate the PCIe addresses into corresponding PCIe addresses for the PCIe devices in the device PCIe address domain.

15. The system of claim 11 , comprising:
responsive to peer-to-peer traffic from a first of the PCIe devices indicating the second of the PCIe devices as a destination in the local PCIe address domain, the isolation function configured to employ a peer-to-peer address trap to receive the peer-to-peer traffic and transfer the peer-to-peer traffic to the second of the PCIe devices in the device PCIe address domain.

16. The system of claim 15 , wherein the peer-to-peer arrangement between the PCIe devices comprises a direct memory access (DMA) link established between memory associated with each of the PCIe devices.

17. The system of claim 11 , comprising:
the control processor configured to establish the isolation function in PCIe switch circuitry that communicatively couples the PCIe devices and the host processor over the PCIe fabric, wherein the isolation function comprises one or more address traps and one or more address translation tables.

18. The system of claim 11 , comprising:
the control processor configured to select PCIe addresses in the isolation function for the PCIe devices in the device PCIe address domain that do not overlap with PCIe address usages of further PCIe devices communicatively coupled on the PCIe fabric.

19. An apparatus comprising:
one or more computer readable storage media;
a processing system operatively coupled with the one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media, that when executed by the processing system, direct the processing system to at least:
establish a peer-to-peer arrangement between Peripheral Component Interconnect Express (PCIe) devices over a PCIe fabric by at least providing an isolation function in the PCIe fabric configured to isolate a first address domain associated with the PCIe devices from at least a second address domain associated with a host;
wherein the isolation function comprises synthetic devices representing the PCIe devices in the second address domain;
wherein the isolation function is configured to redirect traffic transferred by the host for the PCIe devices in the second address domain for delivery to corresponding ones of the PCIe devices in the first address domain; and
wherein the isolation function is further configured to redirect traffic transferred by a first of the PCIe devices indicating the second of the PCIe devices as a destination in the second address domain to the second of

Assignees

• Liqid Inc

Inventors

• Cannata James Scott
• Kazakov German
• Long Christopher R
• Breakstone Jason

Classifications

• G06F13/28

  using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) · CPC title

• G06F13/4282 Primary

  on a serial bus, e.g. I2C bus, SPI bus (on daisy chain buses G06F13/4247) · CPC title

• G06F12/02

  Addressing or allocation; Relocation (program address sequencing G06F9/00; arrangements for selecting an address in a digital store G11C8/00) · CPC title

• G06F9/5077

  Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455) · CPC title

• G06F9/5044

  considering hardware capabilities · CPC title