Scalable MCTP infrastructure

Patent metadata

Field	Value
Publication number	US-11960439-B2
Application number	US-202217690950-A
Country	US
Kind code	B2
Filing date	Mar 9, 2022
Priority date	Mar 9, 2022
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

Methods and apparatus for scalable MCTP infrastructure. A system is split into independent MCTP domains, wherein each MCTP domain uses Endpoint Identifiers (EIDs) for endpoint devices within the MCTP domain in a manner similar to conventional MCTP operations. A new class of MCTP devices (referred to as a Domain Controllers) is provided to enable inter-domain communication and communication with global devices. Global traffic originators or receivers like a BMC (Baseboard Management Controller), Infrastructure Processing Unit (IPU), Smart NIC (Network Interface Card), Debugger, or PROT (Platform Root or Trust) discover and establish two-way communication through the Domain Controllers to any of the devices in the target domain(s). The Domain Controllers are configured to implement tunneled connections between global devices and domain endpoint devices. The tunneled connections may employ encapsulated messages with outer and inner headers and/or augmented MCTP messages with repurposed fields used to store source and destination EIDs.

First claim

What is claimed is:

1. A computer system, comprising:
a plurality of processors, a respective processor including a respective domain controller and coupled to a respect set of one or more domain endpoint devices via respective links;
one or more interconnects, coupled between pairs of processors; and
one or more non-domain endpoint devices communicatively coupled either directly or indirectly to at least one processor,
wherein the computer system is configured to implement multiple MCTP (Management Control Transport Protocol) domains, each MCTP domain employing at least one domain controller and employing 8-bit device Endpoint Identifiers (EIDs) to uniquely identify domain endpoint devices within the MCTP domain, and
wherein the one or more non-domain endpoint devices are enabled to send data to and receive data from domain endpoint devices in each of the multiple MCTP domains.

2. The computer system of claim 1 , wherein the domain controllers and the one of the one or more non-domain endpoint devices are associated with a unique 8-bit EID.

3. The compute system of claim 1 , wherein the one or more non-domain endpoint devices include a one or more of a BMC (Baseboard Management Controller), Infrastructure Processing Unit (IPU), Smart NIC (Network Interface Controller), Debugger, or PROT (Platform Root or Trust).

4. The compute system of claim 3 , wherein the BMC includes a domain controller that is coupled to one or more domain endpoint devices via a respective direct link between the BMC and the domain endpoint device.

5. The compute system of claim 1 , wherein the non-domain endpoint devices comprise a global endpoint device, and wherein communication between domain endpoint devices and non-domain endpoint devices is implemented using messages having a format comprising:
an outer header including a destination EID field and a source EID field; and
an MCTP packet including an inner header having a destination EID field for a destined domain endpoint device and a source EID field for a source domain endpoint device.

6. The compute system of claim 1 , wherein communication between domain endpoint devices and non-domain endpoint devices is implemented using messages employing physical MCTP addressing using one of more of PCIe Bus:Device:Function notation and Route to Root Complex routing.

7. The compute system of claim 1 , wherein the non-domain endpoint devices comprise a global endpoint device, and wherein communication between domain endpoint devices and non-domain endpoint devices is implemented using messages comprising an augmented MCTP packet having a global destination EID stored in a destination endpoint ID field and an in-domain EID for a source or destination domain endpoint device stored in a source endpoint ID field.

8. The compute system of claim 7 , wherein the augmented MCTP packet further includes a Tunnel ID or Domain ID stored in an MCTP reserved field.

9. The compute system of claim 1 , wherein the non-domain endpoint devices comprise a global endpoint device, and wherein communication from a global endpoint device destined to a domain endpoint device is implemented using a message comprising an augmented MCTP packet having a global destination EID for a domain controller and a Tunnel ID stored in a destination endpoint ID field and an in-domain EID for the domain endpoint device stored in a source endpoint ID field,
wherein the message is sent from the global endpoint device to the domain controller having the global destination EID, and wherein the message is forwarded from the domain controller to the domain endpoint device.

10. The compute system of claim 1 , wherein the non-domain endpoint devices comprise a global endpoint device, and wherein communication from a source domain endpoint device destined for a global endpoint device is implemented using a message comprising an augmented MCTP packet having a global destination EID for the destined global endpoint device and a Domain ID stored in a destination endpoint ID field and an in-domain EID for the domain endpoint device stored in a source endpoint ID field,
wherein the message is sent from the source domain endpoint device to a domain controller, and wherein the message is forwarded from the domain controller to the destined global endpoint device.

11. The compute system of claim 1 , wherein the plurality of processors include one or more of a Central Processing Unit (CPU), a Graphic Processor Unit (GPU), a Tensor Processing Unit (TPU), a Data Processor Unit (DPU), an Infrastructure Processing Unit (IPU), an Artificial Intelligence (AI) processor, an AI inference unit, and a Field Programmable Gate Array (FPGA).

12. A method implemented by a system including a plurality of processors, each processor including a respective domain controller and coupled to a respective set of one or more domain endpoint devices via respective links, the system further including one or more non-domain endpoint devices, respective non-domain endpoint devices communicatively coupled either directly or indirectly to at least one processor, the method comprising:
implementing multiple MCTP (Management Control Transport Protocol) domains, each MCTP domain employing a domain controller in a respective processor and employing 8-bit device Endpoint Identifiers (EIDs) to uniquely identify domain endpoint devices within the MCTP domain; and
enabling the one or more non-domain endpoint devices to communicate with domain endpoint devices in each of the multiple MCTP domains.

13. The method of claim 12 , further comprising:
implementing a tunneled connection between a non-domain endpoint device and a domain endpoint device; and
transferring tunneled messages between the non-domain endpoint device and the domain endpoint device using the tunneled connection,
wherein the tunneled connection traverses the domain controller in the processor associated with the MCTP domain including the domain endpoint device.

14. The method of claim 13 , wherein the non-domain endpoint device comprises a global endpoint device, and wherein the messages have a format comprising:
an outer header including a global destination EID field and a global source EID field; and
an MCTP packet including an inner header having a destination EID field for a destined domain endpoint device and a source EID field for a source domain endpoint device;
wherein messages sent from the global endpoint device to the domain endpoint device employ the global source EID.

15. The method of claim 13 , wherein the non-domain endpoint device comprises a global endpoint device, and wherein the messages comprise an augmented MCTP packet having a global destination EID stored in a destination endpoint ID field and an in-domain EID for a source or destination domain endpoint device stored in a source endpoint ID field.

16. The method of claim 15 , wherein the augmented MCTP packet further includes a Tunnel ID or Domain ID stored in an MCTP reserved field.

17. The method of claim 13 , wherein the non-domain endpoint device comprises a global endpoint device, and wherein messages transferred from the global endpoint device destined to the domain endpoint device comprise an augmented MCTP packet having a global destination EID for a domain controller and a Tunnel ID stored in a destination endpoint ID field and an in-domain EID for the domain endpoint device stored in a source endpoint ID field,
wherein the messages are sent from the global endpoint device to the domain controller having the global destination EID, and wherein the message is forwarded from the domain control

Assignees

• Intel Corp

Inventors

• Jurski Janusz
• Loewen Myron
• Oriol Mariusz
• SCHOELLER Patrick
• Backer Jerry
• Thomaiyar Richard Marian
• Louzoun Eliel
• Matuszczak Piotr

Classifications

• G06F15/82 Primary

  data or demand driven · CPC title

• G06F21/6218

  to a system of files or objects, e.g. local or distributed file system or database · CPC title

• G06F21/73 Primary

  by creating or determining hardware identification, e.g. serial numbers · CPC title

• G06F15/177

  Initialisation or configuration control {(processor initialisation G06F9/4405)} · CPC title