Hardware accelerator architecture for processing very-sparse and hyper-sparse matrix data

Patent metadata

Field	Value
Publication number	US-10146738-B2
Application number	US-201615396511-A
Country	US
Kind code	B2
Filing date	Dec 31, 2016
Priority date	Dec 31, 2016
Publication date	Dec 4, 2018
Grant date	Dec 4, 2018

Abstract

An accelerator architecture for processing very-sparse and hyper-sparse matrix data is disclosed. A hardware accelerator comprises one or more tiles, each including a plurality of processing elements (PEs) and a data management unit (DMU). The PEs are to perform matrix operations involving very- or hyper-sparse matrices that are stored by a memory. The DMU is to provide the plurality of PEs access to the memory via an interface that is optimized to provide low-latency, parallel, random accesses to the memory. The PEs, via the DMU, perform the matrix operations by, issuing random access read requests for values of the one or more matrices, issuing random access read requests for values of one or more vectors serving as a second operand, and issuing random access write requests for values of one or more vectors serving as a result.

First claim

What is claimed is:

1. A hardware accelerator comprising:
one or more tiles, wherein each tile includes:
a plurality of processing elements (PEs) to perform matrix operations involving, as a first operand, one or more very- or hyper-sparse matrices that are stored by a memory; and
a data management unit (DMU) to provide the plurality of PEs access to the memory, the memory to be coupled with the hardware accelerator via an interface that is optimized to provide low-latency, parallel, random accesses to data;

wherein the plurality of PEs, via the DMU, perform the matrix operations by,
issuing a first set of random access read requests for values of the one or more matrices after identifying locations of the values by issuing random access read requests for pointer values;
issuing a second set of random access read requests for values of a first set of one or more vectors serving as a second operand; and
issuing a third set of random access write requests for values of a second set of one or more vectors serving as a result.

2. The hardware accelerator of claim 1 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.

3. The hardware accelerator of claim 1 , wherein the memory is a system memory also utilized by a hardware processor.

4. The hardware accelerator of claim 1 , wherein the hardware accelerator is to perform the matrix operations responsive to an offload of one or more tasks issued by hardware processor.

5. The hardware accelerator of claim 1 , wherein the one or more matrices are stored in a compressed format.

6. The hardware accelerator of claim 1 , wherein the matrix operations include multiplication operations.

7. The hardware accelerator of claim 1 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.

8. A method in a hardware accelerator for performing matrix operations with very-sparse or hyper-sparse matrices comprising:
issuing, by one or more processing elements (PEs) of a plurality of PEs of one or more tiles, a first set of random access read requests via one or more data management units (DMUs) to a memory for values of one or more very-sparse or hyper-sparse matrices after identifying locations of the values by issuing random access read requests for pointer values, wherein the one or more DMUs access the memory via an interface that is optimized to provide low-latency, parallel, random accesses to data;
issuing, by the one or more PEs via the one or more DMUs, a second set of random access read requests for values of a first set of one or more vectors serving as an operand; and
issuing, by the one or more PEs via the one or more DMUs, a third set of random access write requests for values of a second set of one or more vectors serving as a result.

9. The method of claim 8 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.

10. The method of claim 8 , wherein the memory is a system memory also utilized by a hardware processor.

11. The method of claim 8 , wherein the issuing the first set of requests, second set of requests, and third set of requests occurs responsive to an offload of one or more tasks by a hardware processor to the hardware accelerator.

12. The method of claim 8 , wherein the one or more matrices are stored in a compressed format.

13. The method of claim 8 , wherein the matrix operations include multiplication operations.

14. The method of claim 8 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.

15. A system comprising:
a memory;
one or more tiles, wherein each tile includes:
a plurality of processing elements (PEs) to perform matrix operations involving, as a first operand, one or more very- or hyper-sparse matrices that are stored by the memory; and
a data management unit (DMU) to provide the plurality of PEs access to the memory, the memory coupled with the hardware accelerator via an interface that is optimized to provide low-latency, parallel, random accesses to data;

wherein the plurality of PEs, via the DMU, perform the matrix operations by,
issuing a first set of random access read requests for values of the one or more matrices after identifying locations of the values by issuing random access read requests for pointer values;
issuing a second set of random access read requests for values of a first set of one or more vectors serving as a second operand; and
issuing a third set of random access write requests for values of a second set of one or more vectors serving as a result.

16. The system of claim 15 , wherein the DMU comprises a cache to store data returned responsive to the issued first set of random access read requests for values of the one or more matrices.

17. The system of claim 15 , wherein the memory is a system memory also utilized by a hardware processor.

18. The system of claim 15 , wherein the system is to perform the matrix operations responsive to an offload of one or more tasks issued by a hardware processor.

19. The system of claim 15 , wherein the one or more matrices are stored in a compressed format.

20. The system of claim 15 , wherein the matrix operations include scale and update operations, multiplication operations, and dot product operations.

Assignees

• Intel Corp

Inventors

• Nurvitadhi Eriko
• Marr Deborah

Classifications

• G11C7/1075

  for multiport memories each having random access ports and serial ports, e.g. video RAM · CPC title

• G06F9/3001 Primary

  Arithmetic instructions · CPC title

• G06F17/16 Primary

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06F15/8069 Primary

  using a cache · CPC title

• G11C7/1072

  for memories with random access ports synchronised on clock signal pulse trains, e.g. synchronous memories, self timed memories · CPC title