Conflict-free, stall-free, broadcast network on chip

Patent metadata

Field	Value
Publication number	US-12406186-B2
Application number	US-202017076287-A
Country	US
Kind code	B2
Filing date	Oct 21, 2020
Priority date	Oct 21, 2020
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Conflict-free, stall-free, broadcast networks on neural inference chips are provided. In various embodiments, a neural inference chip comprises a plurality of network nodes and a network on chip interconnecting the plurality of network nodes. The network comprises at least one pair of directional paths. The paths of each pair have opposite directions and a common end. The network is configured to accept data at any of the plurality of nodes. The network is configured to propagate data along a first of the pair of directional paths from a source node to the common end of the pair of directional paths and along a second of the pair of directional paths from the common end of the pair of directional paths to one or more destination node.

First claim

What is claimed is:

1. A neural inference chip comprising:
a plurality of network nodes organized into a grid of two or more dimensions with at least one row and at least one column;
a network on chip interconnecting the plurality of network nodes, the network comprising a pair of directional paths for each row and a pair of directional paths for each column, the paths of each pair having opposite directions and a common end, the common end of each pair of directional paths being disposed at an end of the grid, wherein
the network is configured to accept data at any of the plurality of nodes,
the network is configured to propagate the data along a first of a pair of directional paths from a source node to the common end of the pair of directional paths, the first of the pair of directional paths traversing a subset of the plurality of network nodes, and
the network is configured to propagate the data along a second of the pair of directional paths from the common end of the pair of directional paths to one or more destination nodes responsive to the data being propagated along the first of the pair of directional paths to the common end, the second of the pair of directional paths traversing the subset of the plurality of network nodes.

2. The neural inference chip of claim 1 , wherein the network is configured to deliver data to the one or more destination node on a second one of the pair of directional paths, according to a network address.

3. The neural inference chip of claim 1 , wherein each network node comprises a router interconnecting a row network and a column of that network node.

4. The neural inference chip of claim 1 , wherein the two or more dimensions are ordered such that for each dimension, data propagates along each of its pair of directional paths followed by a subsequent dimension.

5. The neural inference chip of claim 4 , wherein each network node comprises a router interconnecting the pair of directional paths of each dimension with the pair of directional paths of the subsequent dimension, the router adapted to deliver data from the second of the pair of directional paths to the first of the pair of directional paths of the subsequent dimension.

6. The neural inference chip of claim 1 , further comprising a router at the common end of each pair of directional paths, the router configured to receive data along a first of the pair of directional paths and send data along a second of the pair of directional paths.

7. The neural inference chip of claim 1 , wherein the common end of each pair is configurable at runtime.

8. The neural inference chip of claim 7 , wherein the network is partitioned into a plurality of subpartitions, each subpartition operating as an independent network on a subset of the network nodes.

9. The neural inference chip of claim 1 , wherein the network is configured to accept one packet of data per node per clock cycle, and to propagate data to an adjacent node along each directional path per clock cycle.

10. The neural inference chip of claim 9 , wherein the network is configured to propagate data between adjacent network nodes at every clock cycle without stopping or stalling.

11. The neural inference chip of claim 1 , wherein the network is configured to deliver data by broadcast to all nodes on the second directional path.

12. The neural inference chip of claim 1 , wherein the network is configured to deliver data by multicast to a subset of nodes on the second directional path.

13. The neural inference chip of claim 1 , wherein the network is configured to deliver data by unicast to a single node on the second directional path.

14. The neural inference chip of claim 1 , further comprising an external interface adapted to send and receive data from sources and destinations other than the network nodes.

15. The neural inference chip of claim 14 , wherein the node routers or common end routers are configured to route data bound to the external interface.

16. A method comprising:
accepting data at a plurality of network nodes, the plurality of network nodes being organized into a grid of two or more dimensions with at least one row and at least one column, the plurality of network nodes being interconnected by a network on chip, the network comprising a pair of directional paths for each row and a pair of directional paths for each column, the paths of each pair having opposite directions and a common end, the common end of each pair of directional paths being disposed at an end of the grid;
propagating data along a first of a pair of directional paths from a source node to the common end of the pair of directional paths, the first of the pair of directional paths traversing a subset of the plurality of network nodes; and
propagating, responsive to the data being propagated along the first of the pair of directional paths to the common end, the data along a second of the pair of directional paths from the common end of the pair of directional paths to one or more destination nodes, the second of the pair of directional paths traversing the subset of the plurality of network nodes.

17. The method of claim 16 , further comprising:
propagating multiple adjacent pieces of data over the network on chip, wherein a plurality of source nodes inject data onto the network in a given cycle.

18. The method of claim 16 , further comprising:
propagating multiple adjacent pieces of data over the network on chip, wherein a plurality of source nodes inject data onto the network such that the data is interleaved on the network.

19. The method of claim 16 , further comprising:
propagating multiple adjacent pieces of data over the network on chip, wherein a plurality of source nodes inject data onto different networks of a first dimension such that the data is later interleaved on a network of a second dimension.

20. The method of claim 16 , wherein the injection of data onto the network from one or more source nodes proceeds according to a predetermined schedule.

21. The method of claim 20 , wherein the predetermined schedule ensures that data collisions do not occur on the one or more networks.

Assignees

• IBM

Inventors

• Cassidy Andrew Stephen
• Appuswamy Rathinakumar
• Arthur John Vernon
• Sawada Jun
• Modha Dharmendra S
• Debole Michael Vincent
• Datta Pallab
• Nayak Tapan Kumar

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/048

  Activation functions · CPC title