Chip-to-chip interconnect with a layered communication architecture

Patent metadata

Field	Value
Publication number	US-12061939-B2
Application number	US-202217824766-A
Country	US
Kind code	B2
Filing date	May 25, 2022
Priority date	May 25, 2022
Publication date	Aug 13, 2024
Grant date	Aug 13, 2024

Abstract

A system includes a first integrated circuit package including a first group of one or more artificial intelligence processing units and a first chip-to-chip interconnect communication unit and a second integrated circuit package including a second group of one or more artificial intelligence processing units and a second chip-to-chip interconnect communication unit. The system also includes an interconnect between the first integrated circuit package and the second integrated circuit package, wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit manage ethernet-based communication via the interconnect using a layered communication architecture supporting a credit-based data flow control and a retransmission data flow control.

First claim

What is claimed is:

1. A system, comprising:
a first integrated circuit package including a first group of one or more artificial intelligence processing units and a first chip-to-chip interconnect communication unit;
a second integrated circuit package including a second group of one or more artificial intelligence processing units and a second chip-to-chip interconnect communication unit; and
an interconnect between the first integrated circuit package and the second integrated circuit package, wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit manage ethernet-based communication via the interconnect using a layered communication architecture supporting a credit-based data flow control that updates a total blocks sent counter based on a block size of a sent packet, wherein the interconnect between the first integrated circuit package and the second integrated circuit package includes a direct chip-to-chip connection configured to carry the ethernet-based communication as a direct chip-to-chip communication between the first integrated circuit package as a first chip and the second integrated circuit package as a second chip.

2. The system of claim 1 , wherein the first group of one or more artificial intelligence processing units and the second group of one or more artificial intelligence processing units are configured to handle an artificial intelligence workload that includes training one or more neural networks.

3. The system of claim 1 , wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit include electronic circuitry logic configured to support a physical layer network communication protocol.

4. The system of claim 3 , wherein the physical layer network communication protocol includes an Ethernet communication protocol.

5. The system of claim 1 , wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit include electronic circuitry logic configured to support a plurality of communication protocols beyond a physical layer network communication protocol.

6. The system of claim 1 , wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit include a hardware buffer configured to store communication data.

7. The system of claim 1 , wherein the interconnect includes a physical interconnection between the first integrated circuit package and the second integrated circuit package.

8. The system of claim 7 , wherein the physical interconnection includes an electric wire interconnection.

9. The system of claim 1 , wherein the layered communication architecture includes an application layer.

10. The system of claim 9 , wherein the layered communication architecture further includes a physical network layer.

11. The system of claim 10 , wherein the layered communication architecture further includes a reliability layer.

12. The system of claim 11 , wherein the layered communication architecture further includes a message dispatch and reassembly layer.

13. The system of claim 9 , wherein the application layer is configured to access a high-bandwidth memory.

14. The system of claim 1 , wherein the credit-based data flow control is configured to communicate credit limits between the first integrated circuit package and the second integrated circuit package based on receive buffer sizes of the first integrated circuit package and the second integrated circuit package.

15. The system of claim 1 , wherein the credit-based data flow control is configured to communicate and store counts of packets exchanged between the first integrated circuit package and the second integrated circuit package.

16. The system of claim 1 , wherein the layered communication architecture supports a retransmission data flow control and the retransmission data flow control is configured to acknowledge receipt of a packet without error and update a counter that tracks a consecutive number of received packets without an error.

17. The system of claim 1 , wherein the layered communication architecture supports a retransmission data flow control and the retransmission data flow control is configured to communicate receipt of a packet with an error and cause retransmission of the packet with the error.

18. A method, comprising:
configuring a first integrated circuit package including a first group of one or more artificial intelligence processing units and a first chip-to-chip interconnect communication unit;
configuring a second integrated circuit package including a second group of one or more artificial intelligence processing units and a second chip-to-chip interconnect communication unit; and
establishing an interconnect between the first integrated circuit package and the second integrated circuit package, wherein the first chip-to-chip interconnect communication unit and the second chip-to-chip interconnect communication unit manage ethernet-based communication via the interconnect using a layered communication architecture supporting a credit-based data flow control that updates a total blocks sent counter based on a block size of a sent packet, wherein the interconnect between the first integrated circuit package and the second integrated circuit package includes a direct chip-to-chip connection configured to carry the ethernet-based communication as a direct chip-to-chip communication between the first integrated circuit package as a first chip and the second integrated circuit package as a second chip.

19. A system, comprising:
a first integrated circuit package including a first group of one or more artificial intelligence processing units and a first chip-to-chip interconnect communication unit;
a second integrated circuit package including a second group of one or more artificial intelligence processing units and a switch communication unit, wherein the switch communication unit is configured to provide an intermediate connection point between the first integrated circuit package and at least a third integrated circuit package; and
an interconnect between the first integrated circuit package and the second integrated circuit package, wherein the first chip-to-chip interconnect communication unit and the switch communication unit manage ethernet-based communication via the interconnect using a layered communication architecture supporting a credit-based data flow control that updates a total blocks sent counter based on a block size of a sent packet, wherein the interconnect between the first integrated circuit package and the second integrated circuit package includes a direct chip-to-chip connection configured to carry the ethernet-based communication as a direct chip-to-chip communication between the first integrated circuit package as a first chip and the second integrated circuit package as a second chip.

20. The system of claim 19 , wherein the third integrated circuit package includes a third group of one or more artificial intelligence processing units and a second chip-to-chip interconnect communication unit.

Assignees

• Meta Platforms Inc

Inventors

• Kansal Pankaj
• Srinivasan Arvind
• Reddy Harikrishna Madadi
• Hasani Naader

Classifications

• G06F9/544

  Buffers; Shared memory; Pipes · CPC title

• G06F9/3005

  to perform operations for flow control · CPC title

• H04L1/1806

  Go-back-N protocols · CPC title

• H04L1/188

  Time-out mechanisms · CPC title

• H04L47/39

  Credit based · CPC title