Efficient chip-to-chip communications

Patent metadata

Field	Value
Publication number	US-12585604-B2
Application number	US-202318449979-A
Country	US
Kind code	B2
Filing date	Aug 15, 2023
Priority date	Aug 15, 2023
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

In various examples, when a bridge of a chip has received an eviction request from a client of the chip, the bridge may transmit a read request that corresponds to the same cache line to another chip without waiting for an inter-chip completion response for the eviction request. When the read request is received, the bridge may determine whether the eviction request has already been sent to the other chip and transmit the read request based at least on the eviction request being sent to the other chip using an ordered communication network to ensure the communications are received and/or processed by the other chip in an order that maintains memory coherency. Additionally, the chips may process read unique requests without using an inter-chip completion acknowledgement and may process copy back requests by transmitting corresponding copy back write data with the copy back requests.

First claim

What is claimed is:

1 . A method comprising:
receiving, by an interconnect bridge of a first chip, an eviction request from a client of the first chip; determining, by the interconnect bridge of the first chip, a message of a different transaction than the eviction request; based at least on the message corresponding to a same cache line as the eviction request and the eviction request being of an eviction request type, determining, by the interconnect bridge, whether the eviction request has been transmitted to a second chip using an ordered communication fabric between the first chip and the second chip, the ordered communication fabric and the second chip configured to guarantee that the eviction request and the message are completed by the second chip in an order in which the interconnect bridge transmits the eviction request and the message over the ordered communication fabric; and based at least on the determining whether the eviction request has been transmitted and the eviction request being transmitted to the second chip using the ordered communication fabric, transmitting, by the interconnect bridge and without the first chip waiting for the second chip to complete the eviction request, the message to the second chip using the ordered communication fabric.

2 . The method of claim 1 , wherein the interconnect bridge further provides an internal completion acknowledgement of the eviction request to the client without waiting for the second chip to complete the eviction request.

3 . The method of claim 1 , wherein the transmitting is performed without the second chip having completed the eviction request from the first chip and the message comprises a read request to the same cache line.

4 . The method of claim 1 , wherein the eviction request is for the client to relinquish ownership permissions of the same cache line with respect to at least two clients, the second chip uses the message from the first chip to push the eviction request, received from the first chip, to a second client of the at least two clients, the second client is on the second chip.

5 . The method of claim 1 , wherein the message includes one or more of a read request or a snoop response.

6 . The method of claim 1 , further comprising the interconnect bridge of the first chip deallocating the same cache line from the client based at least on the receiving of the eviction request from the client of the first chip.

7 . The method of claim 1 , wherein the message is based at least on a snoop request and the transmitting of the message to the second chip is based at least on the interconnect bridge determining a completion response for the eviction request has not yet been transmitted to the client of the first chip.

8 . The method of claim 1 , wherein the message is based at least on a snoop request and the interconnect bridge of the first chip retrieves the message from the client of the first chip based at least on determining the interconnect bridge has already transmitted a completion response for the eviction request to the client of the first chip.

9 . The method of claim 1 , wherein the ordered communication fabric is implemented using one or more of a point-to-point communication network or sequence numbers assigned to network communications.

10 . The method of claim 1 , further comprising the interconnect bridge transmitting, to the second chip, a copy back request with copy back write data corresponding to the copy back request.

11 . The method of claim 1 , further comprising the interconnect bridge of the second chip:
transmitting an inter-chip completion data response for a read unique request from the first chip to the first chip; and providing an internal completion acknowledgement for the read unique request to a client of the second chip without waiting for an inter-chip completion acknowledgement for the inter-chip completion data response from the first chip.

12 . At least one processor comprising:
one or more circuits of a first chip to:
receive a request to relinquish, from a client, ownership permissions of a portion of memory with respect to at least two clients;
receive a message corresponding to the portion of memory;
determine the message and the request to relinquish ownership permissions correspond to the portion of memory; and
based at least on the message and the request corresponding to the portion of memory and the request being of an eviction request type, push, over an ordered communication fabric and without the first chip waiting for a second chip to complete the request, the request to relinquish ownership of the portion of memory to the second chip using the message, the ordered communication fabric and the second chip configured to guarantee that the request and the message are completed by the second chip in an order in which the first chip transmits the request and the message over the ordered communication fabric.

13 . The at least one processor of claim 12 , wherein the one or more circuits are to transmit the message to the second chip without the first chip having received a completion response for the request from the second chip.

14 . The at least one processor of claim 12 , where the second chip uses the message from the first chip to push the request, received from the first chip, to a client of the second chip.

15 . The at least one processor of claim 12 , wherein the at least one processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing generative AI operations; a system for performing operations using a large language model; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

16 . A system comprising:
one or more processing units to perform operations including pushing, from a first chip to a second chip and over an ordered communication fabric, a request to relinquish, from a client, ownership permissions of a portion of memory with respect to at least two clients, the ordered communication fabric and the second chip configured to guarantee that the request and a message that corresponds to a same portion of memory as the request are completed by the second chip in an order in which the first chip transmits the request and the message over the ordered communication fabric and the pushing being based at least on:
receiving the message corresponding to a different transaction type than the request, and the request being of an eviction request type.

17 . The system of claim 16 , wherein the request is transmitted to the second chip without the first chip having received a completion response for the request from the second chip.

18 . The system of claim 16 , where the second chip uses the message from the first chip to push the request, re

Assignees

• Nvidia Corp

Inventors

• Chaudhary Anurag
• Wang Guan
• Kumar Harsh

Classifications

• G06F12/0835

  for main memory peripheral accesses (e.g. I/O or DMA) · CPC title

• G06F15/17337

  Direct connection machines, e.g. completely connected computers, point to point communication networks (coupling between buses G06F13/4004) · CPC title

• G06F13/4027

  using bus bridges (G06F13/4022 takes precedence) · CPC title

• G06F12/0831

  using a bus scheme, e.g. with bus monitoring or watching means · CPC title

• G06F2212/1024

  Latency reduction · CPC title