Multiply-accumulate "0" data gating

Patent metadata

Field	Value
Publication number	US-2020293282-A1
Application number	US-202016833128-A
Country	US
Kind code	A1
Filing date	Mar 27, 2020
Priority date	Apr 28, 2017
Publication date	Sep 17, 2020
Grant date	—

Abstract

In an example, an apparatus comprises a plurality of execution units and logic, at least partially including hardware logic, to gate at least one of a multiply unit or an accumulate unit in response to an input of value zero. Other embodiments are also disclosed and claimed.

First claim

1 . An apparatus comprising:
an instruction cache to receive graphics processing instructions; a general-purpose graphics processing compute block comprising a plurality of graphics processing cores to perform operations to execute the graphics processing instructions; and processing circuitry to:
receive, into at least one of a multiply unit or an accumulate unit, a first inference weight and a second inference weight from a layer of a convolutional neural network; and
negate the second inference weight, bypass the multiply unit, and use a shift register when at least one of the first inference weight or the second inference weight are within a threshold value of a power of two, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

2 . The apparatus of claim 1 , further comprising processing circuitry to:
bypass the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of a one, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

3 . The apparatus of claim 1 , further comprising processing circuitry to:
produce no output from the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of zero.

4 . The apparatus of claim 1 , wherein the multiply unit comprises processing circuitry to:
receive the first inference weight and a first indicator of a first number of valid bits in the first inference weight; receive the second inference weight, and a second indicator of a second number of valid bits in the second inference weight; and multiply only the valid bits in the first inference weight and valid bits in the second inference weight.

5 . The apparatus of claim 1 , further comprising processing circuitry to:
produce no output when at least one of the first inference weight or the second inference weight is within a threshold value of zero.

6 . The apparatus of claim 2 , further comprising processing circuitry to:
bypass the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of a one, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

7 . The apparatus of claim 2 , further comprising processing circuitry to:
negate the second operand and bypass the multiplier when at least one of the first operand or the second operand is a negative one; and treat at least one of the first operand or the second operand as a zero when a value of the first operand or the second operand is within a threshold of zero.

8 . The apparatus of claim 1 , further comprising processing circuitry to:
bypass the multiply unit and use the shift register when at least one of the first operand or the second operand is a power of two; and treat at least one of the first operand or the second operand as a power of two when a value of the first operand or the second operand is within a threshold of a power of two.

9 . The apparatus of claim 2 , further comprising a thread scheduler comprising logic, at least partially including hardware logic, to:
break an input vector into a plurality of segments; and perform a dot product using the plurality of segments.

10 . The apparatus of claim 1 , wherein the plurality of execution units are on a single integrated circuit.

11 . A method, comprising:
receiving graphics processing instructions in an instruction cache of a general purpose graphics processor: performing operations to execute the graphics processing instructions; and receiving, into at least one of a multiply unit or an accumulate unit of the general purpose graphics processor, a first inference weight and a second inference weight from a layer of a convolutional neural network; and negating the second inference weight, bypassing the multiply unit, and using a shift register when at least one of the first inference weight or the second inference weight are within a threshold value of a power of two, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

12 . The method of claim 11 , further comprising:
bypassing the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of a one, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

13 . The method of claim 11 , further comprising processing:
producing no output from the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of zero.

14 . The method of claim 11 , further comprising:
receiving the first inference weight and a first indicator of a first number of valid bits in the first inference weight; receiving the second inference weight, and a second indicator of a second number of valid bits in the second inference weight; and multiplying only the valid bits in the first inference weight and valid bits in the second inference weight.

15 . The method of claim 11 , further comprising:
producing no output when at least one of the first inference weight or the second inference weight is within a threshold value of zero.

16 . The method of claim 13 , further comprising:
bypassing the multiply unit when at least one of the first inference weight or the second inference weight is within a threshold value of a one, such that the multiply unit performs no operations on the first inference weight or the second inference weight.

17 . The method of claim 12 , further comprising:
negating the second operand and bypass the multiplier when at least one of the first operand or the second operand is a negative one.

18 . The method of claim 11 , further comprising:
bypassing the multiply unit and use a shift register when at least one of the first operand or the second operand is a power of two; and treating at least one of the first operand or the second operand as a zero when a value of the first operand or the second operand is within a threshold of zero.

19 . The method of claim 11 , further comprising processing circuitry to:
treat at least one of the first operand or the second operand as a power of two when a value of the first operand or the second operand is within a threshold of a power of two.

20 . The method of claim 12 , further comprising:
breaking an input vector into a plurality of segments; and performing a dot product using the plurality of segments.

Assignees

• Intel Corp

Inventors

• Fais Yaniv
• Bar-On Tomer
• Subag Jacob
• Dreyfuss Jeremie
• Faivishevsky Lev
• Behar Michael
• Bleiweiss Amit
• Jacob Guy
• Leibovich Gal
• Ben-Ari Itamar
• Ryvchin Galina
• Yaacoby Eyal

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06F9/3887

  controlled by a single instruction for multiple data lanes [SIMD] · CPC title

• G06F9/5066

  Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451) · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title