Distributed matrix multiplication for neural networks

Patent metadata

Field	Value
Publication number	US-10169296-B2
Application number	US-201615395527-A
Country	US
Kind code	B2
Filing date	Dec 30, 2016
Priority date	Dec 30, 2016
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

In one embodiment, a matrix operation associated with a plurality of input matrices may be performed. The plurality of input matrices may be partitioned into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements. The plurality of input partitions may be distributed among a plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements. A plurality of partial matrix operations may be performed using the plurality of processing elements, and partial matrix data may be transmitted between the plurality of processing elements while performing the plurality of partial matrix operations. A result of the matrix operation may be determined based on the plurality of partial matrix operations.

First claim

What is claimed is:

1. An apparatus, comprising:
a plurality of memory elements to store matrix data, wherein the matrix data comprises a plurality of input matrices; and
a plurality of processing elements to perform a matrix operation associated with the plurality of input matrices, wherein the plurality of processing elements is configured to:
partition the plurality of input matrices into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements;
distribute the plurality of input partitions among the plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements;
perform a plurality of partial matrix operations using the plurality of processing elements;
transmit partial matrix data between the plurality of processing elements while performing the plurality of partial matrix operations; and
determine a result of the matrix operation based on the plurality of partial matrix operations.

2. The apparatus of claim 1 , wherein the matrix operation comprises one or more matrix multiplication operations.

3. The apparatus of claim 1 , wherein:
the plurality of processing elements is configured in a hierarchical arrangement comprising a plurality of processing levels; and
the plurality of processing elements is further configured to distribute the matrix operation across the plurality of processing levels.

4. The apparatus of claim 3 , wherein the plurality of processing elements comprises:
a plurality of matrix processing chips; and
a plurality of matrix processing clusters associated with each matrix processing chip.

5. The apparatus of claim 1 , wherein the plurality of processing elements is further configured to partition the plurality of input matrices based on a number of rows of the plurality of input matrices.

6. The apparatus of claim 1 , wherein:
the plurality of processing elements is configured in a cyclic arrangement such that each processing element is communicatively coupled to a plurality of neighbor processing elements; and
the plurality of neighbor processing elements of each processing element comprises a first neighbor processing element and a second neighbor processing element.

7. The apparatus of claim 6 , wherein the plurality of processing elements is further configured to:
perform the plurality of partial matrix operations in a plurality of stages; and
transmit a portion of the partial matrix data from each processing element to one or more of the neighbor processing elements while performing a particular stage of the partial matrix operations.

8. The apparatus of claim 7 , wherein the plurality of processing elements is further configured to transmit the portion of the partial matrix data from each processing element to the first neighbor processing element and the second neighbor processing element.

9. The apparatus of claim 8 , wherein the partial matrix data comprises a partial input matrix, wherein the partial input matrix is to be used by a first processing element in a particular stage of the partial matrix operations, and wherein the partial input matrix is to be used by a second processing element in a subsequent stage of the partial matrix operations.

10. The apparatus of claim 9 , wherein the matrix operation is associated with a forward propagation operation in a neural network.

11. The apparatus of claim 9 , wherein the matrix operation is associated with a weight update operation in a neural network.

12. The apparatus of claim 7 , wherein the partial matrix data comprises a partial result matrix determined by a first processing element in a particular stage of the partial matrix operations, and wherein the partial result matrix is to be used by a second processing element in a subsequent stage of the partial matrix operations.

13. The apparatus of claim 12 , wherein the matrix operation is associated with a backward propagation operation in a neural network.

14. A method, comprising:
performing a matrix operation associated with a plurality of input matrices, wherein performing the matrix operation comprises:
partitioning the plurality of input matrices into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements;
distributing the plurality of input partitions among a plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements;
performing a plurality of partial matrix operations using the plurality of processing elements;
transmitting partial matrix data between the plurality of processing elements while performing the plurality of partial matrix operations; and
determining a result of the matrix operation based on the plurality of partial matrix operations.

15. The method of claim 14 , wherein the plurality of input matrices is further partitioned based on a number of rows of the plurality of input matrices.

16. The method of claim 14 , wherein:
the plurality of processing elements is configured in a cyclic arrangement such that each processing element is communicatively coupled to a plurality of neighbor processing elements; and
the plurality of neighbor processing elements of each processing element comprises a first neighbor processing element and a second neighbor processing element.

17. The method of claim 16 , wherein the plurality of partial matrix operations is performed in a plurality of stages, and wherein each processing element transmits a portion of the partial matrix data to one or more of the neighbor processing elements while performing a particular stage of the partial matrix operations.

18. The method of claim 17 , wherein the portion of the partial matrix data is transmitted from each processing element to the first neighbor processing element and the second neighbor processing element.

19. A system, comprising:
a plurality of memory elements to store matrix data, wherein the matrix data comprises a plurality of input matrices;
a plurality of processing elements to perform a matrix operation associated with the plurality of input matrices, wherein the plurality of processing elements comprises:
a host processor;
one or more matrix processing chips;
a plurality of matrix processors associated with the one or more matrix processing chips;

wherein the plurality of processing elements is configured to:
partition the plurality of input matrices into a plurality of input partitions, wherein the plurality of input matrices is partitioned based on a number of available processing elements;
distribute the plurality of input partitions among the plurality of processing elements, wherein each input partition is distributed to a particular processing element of the plurality of processing elements;
perform a plurality of partial matrix operations using the plurality of processing elements;
transmit partial matrix data between the plurality of processing elements while performing the plurality of partial matrix operations; and
determine a result of the matrix operation based on the plurality of partial matrix operations.

20. The system of claim 19 , further comprising a communication interface to communicate with one or more remote matrix processing chips over a communication network.

21. At least one non-transitory machine accessible storage medium having instructio

Assignees

• Intel Corp

Inventors

• Korthikanti Vijay Anand R
• Kloss Carey K
• Kalaiah Aravind
• Khosrowshahi Amir

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title