LSTM circuit with selective input computation

Patent metadata

Field	Value
Publication number	US-11790217-B2
Application number	US-201916583201-A
Country	US
Kind code	B2
Filing date	Sep 25, 2019
Priority date	Sep 25, 2019
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

An apparatus is described. The apparatus includes a long short term memory (LSTM) circuit having a multiply accumulate circuit (MAC). The MAC circuit has circuitry to rely on a stored product term rather than explicitly perform a multiplication operation to determine the product term if an accumulation of differences between consecutive, preceding input values has not reached a threshold.

First claim

The invention claimed is:

1. An apparatus, comprising:
a long short term memory (LSTM) circuit comprising a multiply accumulate circuit (MAC) having a multiplier circuit and an accumulator circuit, the LSTM circuit further comprising selection circuitry and a storage element, the selection circuitry to select a stored product term and the multiplier circuit to avoid explicit multiplication of a product term if an accumulation of differences between consecutive, preceding input values has not reached a threshold, the storage element to store the stored product term.

2. The apparatus of claim 1 wherein the MAC circuit is coupled to a sigmoid function block.

3. The apparatus of claim 2 wherein the sigmoid function block is to determine any of a forget, input and output gate of the LSTM circuit.

4. The apparatus of claim 1 wherein the MAC circuit is coupled to a tanh function block.

5. The apparatus of claim 4 wherein an output of the tanh function block is coupled to a multiplier, the multiplier to multiply the output of the tanh function and an input gate value of the LSTM circuit.

6. The apparatus of claim 1 wherein the MAC circuit is a serial MAC circuit that executes multiplications of product terms over a series of cycles within the MAC.

7. The apparatus of claim 1 wherein the MAC circuit is a parallel MAC that executes multiplications of product terms in parallel.

8. The apparatus of claim 1 wherein the multiplier circuit is to explicitly perform a multiplication operation to determine another product term if another accumulation of differences between consecutive, preceding input values has reached the threshold.

9. The apparatus of claim 1 wherein the storage element is at least one of: a memory, a register.

10. The apparatus of claim 1 wherein the apparatus further comprises a register to store the threshold value.

11. A computing system, comprising:
a plurality of processing cores;
a network interface;
a main memory;
a main memory controlled coupled between the main memory and the plurality of processing cores; and,
a long short term memory (LSTM) circuit comprising a multiply accumulate circuit (MAC) having a multiplier circuit and an accumulator circuit, the LSTM circuit further comprising selection circuitry and a storage element, the selection circuitry to select a stored product term and the multiplier circuit to avoid explicit multiplication of a product term if an accumulation of differences between consecutive, preceding input values has not reached a threshold, the storage element to store the stored product term.

12. The computing system of claim 11 wherein the MAC circuit is coupled to a sigmoid function block.

13. The computing system of claim 12 wherein the sigmoid function block is to determine any of a forget, input and output gate of the LSTM circuit.

14. The computing system of claim 11 wherein the MAC circuit is coupled to a tanh function block.

15. The computing system of claim 14 wherein an output of the tanh function block is coupled to a multiplier, the multiplier to multiply the output of the tanh function and an input gate value of the LSTM circuit.

16. The computing system of claim 11 wherein the multiplier circuit is to explicitly perform a multiplication operation to determine another product term if another accumulation of differences between consecutive, preceding input values has reached the threshold.

17. The computing system of claim 16 wherein the storage element is at least one of: a memory, a register.

18. The computing system of claim 11 wherein the computing system further comprises a register to store the threshold value.

19. A method performed by an LSTM circuit, comprising:
determining if an accumulation of differences between input values of consecutive time steps has reached a threshold;
because the accumulation has reached the threshold, selecting with a selection circuit a product term from a storage element for a scalar that is used by the LSTM circuit and not performing a multiplication operation a with a multiplier circuit to determine the product term for the scalar that is used by the LSTM circuit;
determining if another accumulation of differences between input values of other consecutive time steps has reached the threshold;
because the other accumulation has not reached the threshold, performing a multiplication operation with the multiplier circuit to determine another product term for another scalar that is used by the LSTM circuit and not selecting with the selection circuit another product term from the storage element for the another scalar that is used by the LSTM circuit.

20. The method of claim 19 wherein the method is part of a recurring neural network's execution.

Assignees

• Intel Corp

Inventors

• Krishnamurthy Ram
• Chen Gregory K
• Kumar Raghavan
• Knag Phil
• Sumbul Huseyin Ekin

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06F7/5095

  word-serial, i.e. with an accumulator-register · CPC title

• G06F7/523

  Multiplying only · CPC title

• G06F7/5443 Primary

  Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00) · CPC title