Computing efficient cross channel operations in parallel computing machines using systolic arrays

Patent metadata

Field	Value
Publication number	US-11669490-B2
Application number	US-202117518202-A
Country	US
Kind code	B2
Filing date	Nov 3, 2021
Priority date	May 1, 2020
Publication date	Jun 6, 2023
Grant date	Jun 6, 2023

Abstract

An apparatus to facilitate computing efficient cross channel operations in parallel computing machines using systolic arrays is disclosed. The apparatus includes a plurality of registers and one or more processing elements communicably coupled to the plurality of registers. The one or more processing elements include a systolic array circuit to perform cross-channel operations on source data received from a single source register of the plurality of registers, wherein the systolic array circuit is modified to: receive inputs from the single source register at different stages of the systolic array circuit; perform cross-channel operations at channels of the systolic array circuit; bypass disabled channels of the systolic array circuit, the disabled channels not used to compute the cross-channel operations; and broadcast a final result of a final stage of the systolic array circuit to all channels of a destination register.

First claim

What is claimed is:

1. An apparatus comprising:
a plurality of registers; and
one or more processing elements communicably coupled to the plurality of registers, the one or more processing elements comprising:
a systolic array circuit to perform cross-channel operations on source data received from a single source register of the plurality of registers, wherein the systolic array circuit is modified to:
receive inputs from the single source register at different stages of the systolic array circuit, wherein routing circuitry of the different stages is modified to receive the inputs from different channels of the single source register at the different stages of the systolic array circuit;
perform cross-channel operations at channels of the systolic array circuit;
bypass disabled channels of the systolic array circuit, the disabled channels not used to compute the cross-channel operations; and
broadcast a final result of a final stage of the systolic array circuit to all channels of a destination register.

2. The apparatus of claim 1 , wherein the systolic array circuit is modified to broadcast the final result of the final stage further comprises the systolic array circuit to broadcast the final result from a final row of the systolic array circuit to all elements of a destination register of the plurality of registers.

3. The apparatus of claim 1 , wherein the one or more processing elements are comprised in a graphics processing unit (GPU).

4. The apparatus of claim 1 , wherein the systolic array circuit is modified to perform the cross-channel operations by modifying data processing units (DPUs) of the systolic array circuit to perform the cross-channel operations on the source data and modifying routing of the DPUs of the systolic array circuit to receive input from different channels of the single source register at different stages of the systolic array circuit.

5. The apparatus of claim 4 , wherein the different stages of the systolic array circuit each receive a different element of the single source register on which to perform the cross-channel operations.

6. The apparatus of claim 1 , wherein a subset of channels of the systolic array circuit perform the cross-channel operations and wherein other channels of the systolic array circuit that are not comprised in the subset of channels are disabled.

7. The apparatus of claim 1 , wherein the cross-channel operations comprise at least one of a maximum operation, a minimum operation, or an are equal operation.

8. The apparatus of claim 1 , wherein a first channel of a final stage of the systolic array circuit is modified to receive inputs from more than one channel of a previous stage of the systolic array circuit.

9. The apparatus of claim 1 , wherein the apparatus is a single instruction multiple data (SIMD) machine.

10. The apparatus of claim 1 , wherein the apparatus is a single instruction multiple thread (SIMT) machine.

11. A computer-generated method comprising:
receiving, at systolic array hardware circuit modified for cross-channel operations, inputs from a single source register at different stages of the systolic array hardware circuit, wherein routing circuitry of the different stages is modified to receive the inputs from different channels of the single source register at the different stages of the systolic array circuit;
performing cross-channel operations at channels of the systolic array hardware circuit;
bypassing disabled channels of the systolic array hardware circuit, the disabled channels not used to compute the cross-channel operations; and
broadcasting a final result of a final stage of the systolic array hardware circuit to all channels of a destination register.

12. The method of claim 11 , wherein subsequent stages of the systolic array hardware circuit receive a different element of the single source register on which to perform operations.

13. The method of claim 11 , wherein other channels of the systolic array hardware circuit that are not comprised in the disabled channels.

14. The method of claim 11 , wherein the systolic array hardware circuit is part of a graphics processing unit (GPU).

15. The method of claim 11 , wherein a first channel of a final stage of the systolic array circuit is modified to receive inputs from more than one channel of a previous stage of the systolic array hardware circuit.

16. The method of claim 11 , wherein the systolic array hardware circuit is modified for the cross-channel operations by modifying data processing units (DPUs) of the systolic array hardware circuit to perform the cross-channel operations on source data and modifying routing of the DPUs of the systolic array hardware circuit to receive input from different channels of the single source register at different stages of the systolic array hardware circuit.

17. At least one non-transitory computer-readable medium having instructions stored thereon, which when executed by one or more processors, cause the processors to:
receive, at systolic array hardware circuit modified for cross-channel operations, inputs from a single source register at different stages of the systolic array hardware circuit, wherein routing circuitry of the different stages is modified to receive the inputs from different channels of the single source register at the different stages of the systolic array circuit;
perform cross-channel operations at channels of the systolic array hardware circuit;
bypass disabled channels of the systolic array hardware circuit, the disabled channels not used to compute the cross-channel operations; and
broadcast a final result of a final stage of the systolic array hardware circuit to all channels of a destination register.

18. The non-transitory computer-readable medium of claim 17 , wherein subsequent stages of the systolic array hardware circuit receive a different element of the single source register on which to perform operations.

19. The non-transitory computer-readable medium of claim 17 , wherein other channels of the systolic array hardware circuit that are not comprised in the disabled channels.

20. The non-transitory computer-readable medium of claim 17 , wherein the systolic array hardware circuit is modified for the cross-channel operations by modifying data processing units (DPUs) of the systolic array hardware circuit to perform the cross-channel operations on source data and modifying routing of the DPUs of the systolic array hardware circuit to receive input from different channels of the single source register at different stages of the systolic array hardware circuit.

Assignees

• Intel Corp

Inventors

• Maiyuran Subramaniam
• Parra Jorge
• Pal Supratim
• Gurram Chandra

Classifications

• G06N20/00

  Machine learning · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06F15/8046 Primary

  Systolic arrays · CPC title

• G06F15/8007

  single instruction multiple data [SIMD] multiprocessors · CPC title