Integer matrix multiplication engine using pipelining

Patent metadata

Field	Value
Publication number	US-11880426-B2
Application number	US-202217878011-A
Country	US
Kind code	B2
Filing date	Jul 31, 2022
Priority date	Apr 1, 2019
Publication date	Jan 23, 2024
Grant date	Jan 23, 2024

Abstract

Techniques for data manipulation using integer matrix multiplication using pipelining are disclosed. A first integer matrix with dimensions m×k and a second integer matrix with dimensions k×n are obtained for matrix multiplication within a processor. The first and second integer matrices employ a two's complement variable radix point data representation. The first and second integer matrices are distilled into (j×j) submatrices. A first variable radix point format and an initial value for an accumulator register are configured dynamically. A first variable radix point format is configured dynamically for the first integer matrix and a second variable radix point format is configured dynamically for the second integer matrix. Multiply-accumulate operations are executed in a pipelined fashion on the (j×j) submatrices of the first integer matrix and the second integer matrix, where a third variable radix point format is configured for the result.

First claim

What is claimed is:

1. A processor-implemented method comprising:
obtaining a first integer matrix with dimensions m×k and a second integer matrix with dimensions k×n for matrix multiplication within a processor, wherein the first integer matrix and the second integer matrix employ a two's complement variable radix point data representation;
distilling the first integer matrix and the second integer matrix into (j×j) submatrices;
configuring dynamically both a variable radix point format and an initial value for an accumulator register;
executing multiply-accumulate (MAC) operations in a pipelined architecture on the (j×j) submatrices of the first integer matrix and the second integer matrix, wherein a third variable radix point format is configured for the result; and
using the executed MAC operations to process data in a data-flow architecture.

2. The method of claim 1 further comprising configuring dynamically a first variable radix point format for the first integer matrix and a second variable radix point format for the second integer matrix.

3. The method of claim 2 wherein the first variable radix point format and the second variable radix point format comprise a 16-bit data type.

4. The method of claim 2 wherein the first variable radix point format and the second variable radix point format comprise a 4-bit primitive data type.

5. The method of claim 2 wherein the first variable radix point format and the second variable radix point format comprise an 8-bit primitive data type.

6. The method of claim 1 further comprising outputting results of the matrix multiplication to a storage element, wherein the outputting takes an additional (m×k) cycles.

7. The method of claim 1 wherein the first integer matrix and the second integer matrix comprise subsections of an o-dimensional tensor, wherein o is greater than 2.

8. The method of claim 1 wherein each multiply-accumulate (MAC) unit used for matrix multiplication in the processor is configured to have an accumulator depth of m.

9. The method of claim 1 further comprising pipelining input elements to multiply-accumulate (MAC) units used for matrix multiplication in the processor through two input registers.

10. The method of claim 1 wherein performing N multiply-accumulate (MAC) operations in parallel reduces an amount of time taken to perform the N MAC operations from an order of magnitude of N 3 to an order of magnitude of N 2 .

11. The method of claim 1 further comprising adding one or more idle or no operation (NOP) cycles after completion of a matrix multiply operation before starting a next matrix multiply operation.

12. The method of claim 1 wherein a processor and memory subsystem is allocated as part of one or more clusters within a reconfigurable fabric to implement MAC units.

13. The method of claim 12 , wherein each cluster of the one or more clusters within the reconfigurable fabric is controlled by one or more circular buffers.

14. The method of claim 13 , wherein the one or more circular buffers are statically scheduled.

15. The method of claim 12 , wherein each cluster of the one or more clusters within the reconfigurable fabric comprises process elements, switching elements, or storage elements.

16. The method of claim 1 , wherein the data-flow architecture implements machine learning.

17. The method of claim 1 , wherein the machine learning utilizes one or more convolutional neural networks.

18. The method of claim 1 , wherein executing multiply-accumulate operations in a pipelined architecture is accomplished using systolic data flow.

19. One or more non-transitory computer readable media embodying one or more instructions that are operable when executing by one or more processors to perform operations of:
obtaining a first integer matrix with dimensions m×k and a second integer matrix with dimensions k×n for matrix multiplication within a processor, wherein the first integer matrix and the second integer matrix employ a two's complement variable radix point data representation;
distilling the first integer matrix and the second integer matrix into (j×j) submatrices;
configuring dynamically both a variable radix point format and an initial value for an accumulator register;
executing multiply-accumulate operations in a pipelined architecture—on the (j×j) submatrices of the first integer matrix and the second integer matrix, wherein a third variable radix point format is configured for the result; and
using the executed MAC operations to process data in a data-flow architecture.

20. A system comprising: a memory which stores instructions; and
one or more processors coupled to the memory wherein the one or more processors, when executing the instructions, are configured to:
obtain a first integer matrix with dimensions m×k and a second integer matrix with dimensions k×n for matrix multiplication within a processor, wherein the first integer matrix and the second integer matrix employ a two's complement variable radix point data representation;
distill the first integer matrix and the second integer matrix into (j×j) submatrices;
configure dynamically both a variable radix point format and an initial value for an accumulator register; and
execute multiply-accumulate operations in a pipelined architecture on the (j×j) submatrices of the first integer matrix and the second integer matrix, wherein a third variable radix point format is configured for the result; and
use the executed MAC operations to process data in a data-flow architecture.

Assignees

• Wave Computing Inc

Inventors

• Simpson David John

Classifications

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F7/16 Primary

  Combined merging and sorting · CPC title