Reduced power consumption analog or hybrid mac neural network

Patent metadata

Field	Value
Publication number	US-2023244921-A1
Application number	US-202217588657-A
Country	US
Kind code	A1
Filing date	Jan 31, 2022
Priority date	Jan 31, 2022
Publication date	Aug 3, 2023
Grant date	—

Abstract

Power efficient performance may be implemented in a hardware accelerator (e.g., a neural processor) comprising hybrid or analog multiply and accumulate (MAC) processing elements (PEs). For example, power consumption may be reduced in neural networks with a rectified linear unit (ReLU) activation layer. A hybrid or analog MAC circuit may be configured with a look-ahead sign detector to dynamically stop computations prior to completion, for example, based on detection of a negative value, which a ReLU activation layer may (e.g., subsequently) convert to zero. The sign of a value may be indicated by a most significant bit (MSB). A controller may provide power and/or clock cycles to an analog to digital converter (ADC) to determine a sign of a value being computed. The sign may be used to selectively complete computations for positive values and selectively terminate computations for negative values, thereby reducing power consumption of the MAC circuit.

First claim

What is claimed is:

1 . A neural processing unit (NPU) configured to implement an artificial intelligence (AI) neural network (NN) model, the NPU comprising:
a multiply and accumulate (MAC) circuit comprising:
a plurality of MAC processing elements (PEs) configured to perform multiplication operations on inputs and weights to generate mid-term values and to accumulate the mid-term values as an analog output;
an analog to digital converter (ADC) configured to convert the analog output to a digital output; and
an ADC controller configured to selectively stop the ADC before completely converting the analog output to the digital output based at least on a sign of the digital output, thereby reducing power consumption of the MAC circuit.

2 . The NPU of claim 1 , wherein the MAC circuit comprises an analog MAC circuit or a hybrid MAC circuit.

3 . The NPU of claim 1 , wherein the ADC controller is configured to selectively stop the ADC by selectively interrupting a power supply to the ADC.

4 . The NPU of claim 1 , wherein the ADC comprises a successive approximation register (SAR) ADC and wherein the ADC controller is configured to selectively stop the SAR ADC to reduce a number of cycles of the SAR ADC.

5 . The NPU of claim 1 , wherein the ADC controller is configured to selectively stop the ADC based on an indication of a type of activation function applied to the digital output and based on the sign of the digital output.

6 . The NPU of claim 5 , wherein the ADC controller is configured to selectively stop the ADC based on an indication that the activation function applied to the digital output comprises a rectified linear unit (ReLU) activation function and the sign of the digital output is negative.

7 . The NPU of claim 5 , the MAC circuit further comprising:
an activation function detector configured to determine the type of activation function.

8 . The NPU of claim 1 , the MAC circuit further comprising:
a look-ahead sign detector configured to determine the sign of the digital output.

9 . A method of improving implementation of an artificial intelligence (AI) neural network (NN) model, the method comprising:
performing multiply and accumulate (MAC) operations on inputs and weights provided to a MAC circuit; generating an analog output from the MAC operations; partially converting, by an analog to digital converter (ADC), the analog output to a digital output; determining a sign of the digital output based on the partial conversion; completing conversion of the analog output to the digital output by the ADC responsive to at least determining that the sign is a first sign; and reducing power consumption of the MAC circuit by stopping conversion of the analog output to the digital output by the ADC responsive to at least determining that the sign is a second sign, wherein the second sign different from the first sign.

10 . The method of claim 9 , wherein the partially converting comprises converting the analog output to a most significant bit (MSB) of the digital output, and wherein the sign of the digital output is indicated by the MSB.

11 . The method of claim 9 , wherein the stopping the conversion of the analog output to the digital output by the ADC comprises interrupting a power supply to the ADC.

12 . The method of claim 9 , wherein the ADC comprises a successive approximation register (SAR) ADC, and wherein the stopping the conversion of the analog output to the digital output by the ADC reduces the power consumption of the MAC circuit by reducing a number of cycles of the SAR ADC.

13 . The method of claim 9 , wherein stopping the conversion of the analog output to the digital output by the ADC responsive to at least determining that the sign is the second sign comprises:
stopping the conversion of the analog output to the digital output by the ADC responsive to an indication of a first type of activation function applied to the digital output and responsive to determining that the sign is the second sign.

14 . The method of claim 13 , wherein the first type of activation function is a rectified linear unit (ReLU) activation function, the second sign of the digital output is negative and the first sign of the digital output is positive.

15 . The method of claim 13 , further comprising:
determining the type of activation function to be one of the first type of activation function or a second type of activation function.

16 . A system, comprising:
one or more memory devices that store a deep neural network (DNN) topology; and a neural processing unit (NPU) comprising:
a multiply and accumulate (MAC) circuit comprising:
a plurality of MAC processing elements (PEs) configured to generate an analog output from multiply and accumulation operations on inputs and weights;
an analog to digital converter (ADC) configured to convert the analog output to a digital output; and
an ADC controller configured to selectively stop the ADC before completely converting the analog output to the digital output based at least on a sign of the digital output, thereby reducing power consumption of the MAC circuit.

17 . The system of claim 16 , wherein the ADC controller is configured to selectively stop the ADC based on an indication of a type of activation function applied to the digital output and based on the sign of the digital output.

18 . The system of claim 17 , wherein the ADC controller is configured to selectively stop the ADC based on an indication that the activation function comprises a rectified linear unit (ReLU) activation function and the sign of the digital output is negative.

19 . The system of claim 16 , the MAC circuit further comprising:
an activation function detector configured to determine the type of activation function.

20 . The system of claim 16 , the MAC circuit further comprising:
a look-ahead sign detector configured to determine the sign of the digital output.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Royzen Evgeny
• Rogachov Evgeny

Classifications

• G06N3/0635 Primary

  Physics · mapped topic

• G06J1/00

  Hybrid computing arrangements · CPC title

• G06N3/065 Primary

  Analogue means · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title