Distributed matrix multiplication for neural networks

Patent metadata

Field	Value
Publication number	US-10922380-B2
Application number	US-201816236955-A
Country	US
Kind code	B2
Filing date	Dec 31, 2018
Priority date	Dec 30, 2016
Publication date	Feb 16, 2021
Grant date	Feb 16, 2021

Abstract

In one embodiment, a matrix operation associated with a plurality of input matrices may be performed. The plurality of input matrices may be partitioned into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements. The plurality of input partitions may be distributed among a plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements. A plurality of partial matrix operations may be performed using the plurality of processing elements, and partial matrix data may be transmitted between the plurality of processing elements while performing the plurality of partial matrix operations. A result of the matrix operation may be determined based on the plurality of partial matrix operations.

First claim

What is claimed is:

1. A matrix processor, comprising:
a memory to store a plurality of input matrices;
a plurality of matrix processing units (MPUs) to perform matrix multiplication arithmetic;
controller circuitry to:
receive an instruction to be executed by the matrix processor, wherein the instruction instructs the matrix processor to perform a matrix multiplication operation on the plurality of input matrices;
partition the plurality of input matrices into a plurality of input partitions based on a number of available MPUs;
distribute the plurality of input partitions among the plurality of MPUs, wherein each input partition is distributed to a particular MPU of the plurality of MPUs;
perform a plurality of partial matrix multiplication calculations using the plurality of MPUs;
transmit partial matrix data between the plurality of MPUs while performing the plurality of partial matrix multiplication calculations, wherein each MPU is to transmit a portion of the partial matrix data to one or more of the plurality of MPUs simultaneously while each of the plurality of partial matrix multiplication calculations is being performed; and
determine a result of the matrix multiplication operation based on the plurality of partial matrix multiplication calculations.

2. The matrix processor of claim 1 , wherein:
the plurality of MPUs is configured in a cyclic arrangement such that each MPU is communicatively coupled to a plurality of neighbor MPUs; and
the plurality of neighbor MPUs of each MPU comprises a first neighbor MPU and a second neighbor MPU.

3. The matrix processor of claim 2 , wherein the controller circuitry is further to:
perform the plurality of partial matrix multiplication calculations in a plurality of stages; and
transmit a portion of the partial matrix data from each MPU to one or more of the plurality of neighbor MPUs while performing each stage of the plurality of partial matrix multiplication calculations.

4. The matrix processor of claim 3 , wherein the controller circuitry to transmit the portion of the partial matrix data from each MPU to one or more of the plurality of neighbor MPUs while performing each stage of the plurality of partial matrix multiplication calculations is further to:
transmit the portion of the partial matrix data from each MPU to the first neighbor MPU and the second neighbor MPU.

5. The matrix processor of claim 4 , wherein the partial matrix data comprises a partial input matrix, wherein the partial input matrix is to be used by a first MPU in a particular stage of the plurality of partial matrix multiplication calculations, and wherein the partial input matrix is to be used by a second MPU in a subsequent stage of the plurality of partial matrix multiplication calculations.

6. The matrix processor of claim 5 , wherein the matrix multiplication operation is associated with a forward propagation operation in a neural network.

7. The matrix processor of claim 5 , wherein the matrix multiplication operation is associated with a weight update operation in a neural network.

8. The matrix processor of claim 3 , wherein the partial matrix data comprises a partial result matrix determined by a first MPU in a particular stage of the plurality of partial matrix multiplication calculations, and wherein the partial result matrix is to be used by a second MPU in a subsequent stage of the plurality of partial matrix multiplication calculations.

9. The matrix processor of claim 8 , wherein the matrix multiplication operation is associated with a backward propagation operation in a neural network.

10. At least one non-transitory machine accessible storage medium having instructions stored thereon, wherein the instructions, when executed on a matrix processor, cause the matrix processor to:
receive, from a host processor, a request to perform a matrix multiplication operation on a plurality of input matrices;
partition the plurality of input matrices into a plurality of input partitions based on a number of available matrix processing units (MPUs) in the matrix processor;
distribute the plurality of input partitions among a plurality of MPUs in the matrix processor, wherein each input partition is distributed to a particular MPU of the plurality of MPUs;
perform a plurality of partial matrix multiplication calculations using the plurality of MPUs;
transmit partial matrix data between the plurality of MPUs while performing the plurality of partial matrix multiplication calculations, wherein each MPU is to transmit a portion of the partial matrix data to one or more of the plurality of MPUs simultaneously while each of the plurality of partial matrix multiplication calculations is being performed; and
determine a result of the matrix multiplication operation based on the plurality of partial matrix multiplication calculations.

11. The storage medium of claim 10 , wherein:
the plurality of MPUs is configured in a cyclic arrangement such that each MPU is communicatively coupled to a plurality of neighbor MPUs; and
the plurality of neighbor MPUs of each MPU comprises a first neighbor MPU and a second neighbor MPU.

12. The storage medium of claim 11 , wherein the instructions further cause the matrix processor to:
perform the plurality of partial matrix multiplication calculations in a plurality of stages; and
transmit a portion of the partial matrix data from each MPU to one or more of the plurality of neighbor MPUs while performing each stage of the plurality of partial matrix multiplication calculations.

13. The storage medium of claim 12 , wherein the instructions that cause the matrix processor to transmit the portion of the partial matrix data from each MPU to one or more of the plurality of neighbor MPUs while performing each stage of the plurality of partial matrix multiplication calculations further cause the matrix processor to:
transmit the portion of the partial matrix data from each MPU to the first neighbor MPU and the second neighbor MPU.

14. The storage medium of claim 13 , wherein the partial matrix data comprises a partial input matrix, wherein the partial input matrix is to be used by a first MPU in a particular stage of the plurality of partial matrix multiplication calculations, and wherein the partial input matrix is to be used by a second MPU in a subsequent stage of the plurality of partial matrix multiplication calculations.

15. The storage medium of claim 14 , wherein the matrix multiplication operation is associated with a forward propagation operation in a neural network.

16. The storage medium of claim 14 , wherein the matrix multiplication operation is associated with a weight update operation in a neural network.

17. The storage medium of claim 12 , wherein the partial matrix data comprises a partial result matrix determined by a first MPU in a particular stage of the plurality of partial matrix multiplication calculations, and wherein the partial result matrix is to be used by a second MPU in a subsequent stage of the plurality of partial matrix multiplication calculations.

18. The storage medium of claim 17 , wherein the matrix multiplication operation is associated with a backward propagation operation in a neural network.

19. A method of performing matrix multiplication on a matrix processor, comprising:
receiving, from a host processor, a request to perform a matrix multiplication operation on a plurality of input matrices;
partitioning the plurality of input matrices into a plurality of input partitions based on a number of available matrix processin

Assignees

• Intel Corp

Inventors

• Korthikanti Vijay Anand R
• Kloss Carey K
• Kalaiah Aravind
• Khosrowshahi Amir

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title