Variable clock adaptation in neural network processors

Patent metadata

Field	Value
Publication number	US-11900240-B2
Application number	US-202017023144-A
Country	US
Kind code	B2
Filing date	Sep 16, 2020
Priority date	Sep 18, 2019
Publication date	Feb 13, 2024
Grant date	Feb 13, 2024

Abstract

Systems and devices are provided to increase computational and/or power efficiency for one or more neural networks via a computationally driven closed-loop dynamic clock control. A clock frequency control word is generated based on information indicative of a current frame execution rate of a processing task of the neural network and a reference clock signal. A clock generator generates the clock signal of neural network based on the clock frequency control word. A reference frequency may be used to generate the clock frequency control word, and the reference frequency may be based on information indicative of a sparsity of data of a training frame.

First claim

What is claimed is:

1. A device, comprising:
control word generation circuitry, which, in operation, generates a clock frequency control word based on information indicative of a current frame execution rate of a processing task of a neural network and a reference clock signal; and
a clock generator coupled to the control word generation circuitry and which, in operation, generates a neural network clock signal for use by the neural network based on the clock frequency control word.

2. The device of claim 1 , wherein the control word generation circuitry, in operation, determines the frequency control word based on a reference frequency.

3. The device of claim 2 , wherein the information indicative of the current frame execution rate comprises frame synchronization information and the control word generation circuitry comprises:
a frequency counter which, in operation, determines a current frame processing frequency based at least in part on the reference clock signal and the frame synchronization information; and
a proportional integral controller which, in operation, generates the clock frequency control word based on a difference between the determined frame processing frequency and the reference frequency.

4. The device of claim 3 , wherein the reference frequency is based on a frame processing frequency of an execution of a kernel processing only non-zero operations.

5. The device of claim 4 , wherein the reference frequency corresponds to a nominal kernel data sparsity associated with the processing task of the neural network.

6. The device of claim 2 , wherein the reference frequency is based on an average frame execution rate.

7. The device of claim 3 , wherein the control word generating circuitry comprises a frequency error generator coupled to the frequency counter, which, in operation, determines the difference between the determined frame processing frequency and the reference frequency.

8. The device of claim 1 , comprising an adaptive voltage scaling controller which, in operation, modifies a supply voltage of the neural network.

9. The device of claim 8 , wherein, in operation, the adaptive voltage scaling controller modifies the supply voltage based on the neural network clock signal.

10. The device of claim 1 , comprising a dynamic voltage frequency scaling controller which, in operation, generates control information, and the clock generator generates the neural network clock signal based on the control information generated by the dynamic voltage frequency scaling controller.

11. The device of claim 1 , wherein the information indicative of a current frame execution rate includes a plurality of start interrupt times associated with the processing task and a corresponding plurality of stop interrupt times associated with the processing task.

12. The device of claim 11 , wherein each of the plurality of start interrupt times corresponds to a starting time associated with processing a respective frame of data and each of the plurality of stop interrupt times corresponds to a stopping time associated with processing the respective frame of data.

13. The device of claim 12 , wherein each of the plurality of start interrupt times and stop interrupt times is associated with a processing weight, and the control word generation circuitry, in operation, generates the clock frequency control word based on the processing weights.

14. The device of claim 1 , wherein the information indicative of the current frame execution rate is associated with a current kernel data sparsity of the processing task of the neural network.

15. The device of claim 1 , wherein the clock generator is one of a group that includes a phase lock loop (PLL) clock generator and a frequency lock loop (FLL) clock generator.

16. A system, comprising:
neural network accelerator circuitry; and
closed-loop clock-rate control circuitry coupled to the neural network accelerator circuitry, and which, in operation, generates a clock signal of the neural network accelerator circuitry, the closed-loop clock-rate control circuitry including:
control-word generation circuitry, which, in operation, generates a clock frequency control word based on information indicative of a current frame execution rate of a processing task of the neural network accelerator circuitry and a reference clock signal; and
a clock generator coupled to the control word generation circuitry and which, in operation, generates the clock signal of neural network accelerator circuitry based on the clock frequency control word.

17. The system of claim 16 , wherein the control-word generation circuitry, in operation, generates the clock frequency control word based on a reference frequency.

18. The system of claim 16 , comprising:
an adaptive voltage scaling controller, which, in operation, modifies a supply voltage of the neural network.

19. The system of claim 16 , comprising:
a dynamic voltage frequency scaling controller, which, in operation, generates dynamic clock control information, wherein the clock generator generates the clock signal of the neural network accelerator circuitry based on the dynamic clock control information.

20. A method, comprising:
executing a kernel using a neural network accelerator; and
performing closed-loop clock-rate control of an operating frequency of the neural network accelerator during execution of the kernel, the performing closed-loop clock-rate control including:
generating a clock frequency control word based on information indicative of a current frame execution rate of the kernel and a reference clock signal; and
generating a clock signal of neural network accelerator circuitry based on the clock frequency control word.

21. The method of claim 20 , wherein the clock frequency control word is generated based on a reference frequency.

22. The method of claim 20 , comprising:
modifying a supply voltage of the neural network based on the generated clock signal of the neural network accelerator circuitry.

23. The method of claim 20 , comprising:
modifying the operating frequency of the neural network using a dynamic voltage frequency scaling controller.

Assignees

• St Microelectronics Srl
• St Microelectronics Int Nv

Inventors

• Chawla Nitin
• Desoli Giuseppe
• Ayodhyawasi Manuj
• Boesch Thomas
• Singh Surinder Pal

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06F1/08

  Clock generators with changeable or programmable clock frequency · CPC title