Apparatus and method for a masked multiply instruction to support neural network pruning operations

Patent metadata

Field	Value
Publication number	US-10929503-B2
Application number	US-201816230814-A
Country	US
Kind code	B2
Filing date	Dec 21, 2018
Priority date	Dec 21, 2018
Publication date	Feb 23, 2021
Grant date	Feb 23, 2021

Abstract

An apparatus and method for a masked multiply instruction to support neural network pruning operations. For example, one embodiment of a processor comprises: a decoder to decode a matrix multiplication with masking (GEMM) instruction identifying a destination matrix register to store a result, and source registers storing an A-matrix, a B-matrix, and a matrix mask; execution circuitry to execute the GEMM instruction, the execution circuitry to multiply a plurality of B-matrix elements with a plurality of A-matrix elements, each of the B-matrix elements associated with a mask value in the matrix mask, wherein if the mask value is set to a first value, then the execution circuitry is to multiply the B-matrix element with one or more of the A-matrix elements to generate a first partial result, and if the mask value is set to a second value, then the execution circuitry is to multiply an alternate B-matrix element with a one or more of the A-matrix elements to generate a second partial result.

First claim

What is claimed is:

1. A processor comprising:
a decoder to decode a matrix multiplication with masking (GEMM) instruction identifying a destination matrix register to store a result, and source registers storing an A-matrix, at least a portion of a B-matrix, and a matrix mask;
execution circuitry to execute the GEMM instruction, the execution circuitry to multiply a plurality of B-matrix elements with a plurality of A-matrix elements, each of the B-matrix elements associated with a mask value in the matrix mask,
wherein if the mask value is set to a first value, then the execution circuitry is to multiply the B-matrix element with one or more of the A-matrix elements to generate a first partial result, and if the mask value is set to a second value, then the execution circuitry is to multiply an alternate B-matrix element with one or more of the A-matrix elements to generate a second partial result.

2. The processor of claim 1 wherein the plurality of B-matrix elements comprise a B-matrix column and the plurality of A-matrix elements comprise A-matrix rows which are to be multiplied by the B-matrix column when the mask value for each B-matrix element in the B-matrix column is set to the first value, or to substitute one or more B-matrix elements from alternate B-matrix columns when the mask value associated with one or more of the B-matrix elements is set to the second value.

3. The processor of claim 2 wherein the execution circuitry is to select the alternate B-matrix element in accordance with a window size specifying a set of B-matrix columns from which the alternate B-matrix element can be selected.

4. The processor of claim 3 wherein responsive to a first window size, the execution circuitry is to select one or more alternate B-matrix elements from a single alternate B-matrix column and responsive to a second window size the execution circuitry is to select one or more alternate B-matrix elements from a plurality of alternate B-matrix columns.

5. The processor of claim 1 wherein the execution circuitry is to select the alternate B-matrix element from a different column than the B-matrix element.

6. The processor of claim 1 wherein the execution circuitry comprises a set of multiplexers to select the B-matrix element or the alternate B-matrix element responsive to the mask value.

7. The processor of claim 6 wherein the execution circuitry further comprises:
adder reduction circuitry to add and/or perform a reduction operation on a plurality of products generated from the multiplications of the B-matrix elements or alternate B-matrix elements with the A-matrix elements.

8. The processor of claim 1 wherein the source registers comprise a first tile register to store the A-matrix, a second tile register to store the B-matrix, and a matrix mask register to store the mask matrix, and wherein the destination matrix register comprises a destination tile register to store a result of the multiplications performed by the execution circuitry.

9. The processor of claim 8 wherein the first tile register, the second tile register, and the destination tile register each comprise a plurality of vector registers.

10. A method comprising:
decoding a matrix multiplication with masking (GEMM) instruction identifying a destination matrix register to store a result, and source registers storing an A-matrix, a B-matrix, and a matrix mask;
multiplying a plurality of B-matrix elements with a plurality of A-matrix elements responsive to the GEMM instruction, each of the B-matrix elements associated with a mask value in the matrix mask,
wherein if the mask value is set to a first value, then the B-matrix element is multiplied with one or more of the A-matrix elements to generate a first partial result, and
if the mask value is set to a second value, then an alternate B-matrix element is multiplied with one or more of the A-matrix elements to generate a second partial result.

11. The method of claim 10 wherein the plurality of B-matrix elements comprise a B-matrix column and the plurality of A-matrix elements comprise A-matrix rows which are to be multiplied by the B-matrix column when the mask value for each B-matrix element in the B-matrix column is set to the first value, or to substitute one or more B-matrix elements from alternate B-matrix columns when the mask value associated with one or more of the B-matrix elements is set to the second value.

12. The method of claim 11 wherein the alternate B-matrix element is selected in accordance with a window size specifying a set of B-matrix columns from which the alternate B-matrix element can be selected.

13. The method of claim 12 wherein responsive to a first window size, one or more alternate B-matrix elements are selected from a single alternate B-matrix column and responsive to a second window size one or more alternate B-matrix elements are selected from a plurality of alternate B-matrix columns.

14. The method of claim 10 wherein the alternate B-matrix element is selected from a different column than the B-matrix element.

15. The method of claim 10 wherein the B-matrix element or alternate B-matrix element is selected by a set of multiplexers responsive to the mask value.

16. The method of claim 15 further comprising:
adding and/or performing a reduction operation on a plurality of products generated from the multiplications of the B-matrix elements or alternate B-matrix elements with the A-matrix elements.

17. The method of claim 10 further comprising:
storing the A-matrix in a first tile register;
storing the B-matrix elements in a second tile register;
storing the mask matrix in a mask matrix register; and
storing a result of the multiplications in a destination tile register.

18. The method of claim 17 wherein the first tile register, the second tile register, and the destination tile register each comprise a plurality of vector registers.

19. A machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform the operations of:
decoding a matrix multiplication with masking (GEMM) instruction identifying a destination matrix register to store a result, and source registers storing an A-matrix, a B-matrix, and a matrix mask;
multiplying a plurality of B-matrix elements with a plurality of A-matrix elements responsive to the GEMM instruction, each of the B-matrix elements associated with a mask value in the matrix mask,
wherein if the mask value is set to a first value, then the B-matrix element is multiplied with one or more of the A-matrix elements to generate a first partial result, and
if the mask value is set to a second value, then an alternate B-matrix element is multiplied with one or more of the A-matrix elements to generate a second partial result.

20. The machine-readable medium of claim 19 wherein the plurality of B-matrix elements comprise a B-matrix column and the plurality of A-matrix elements comprise A-matrix rows which are to be multiplied by the B-matrix column when the mask value for each B-matrix element in the B-matrix column is set to the first value, or to substitute one or more B-matrix elements from alternate B-matrix columns when the mask value associated with one or more of the B-matrix elements is set to the second value.

21. The machine-readable medium of claim 20 wherein the alternate B-matrix element is selected in accordance with a window size specifying a set of B-matrix columns from which the alternate B-matrix element can be selected.

Assignees

• Intel Corp

Inventors

• Azizi Omid
• Koren Chen
• Garegrat Nitin

Classifications

• G06N3/048

  Activation functions · CPC title

• G06F9/30038

  using a mask · CPC title

• G06F9/30018

  Bit or string instructions · CPC title

• G06F9/30036

  Instructions to perform operations on packed data, e.g. vector, tile or matrix operations · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title