Methods and systems for online selection of number formats for network parameters of a neural network

Patent metadata

Field	Value
Publication number	US-2024160908-A1
Application number	US-202318375198-A
Country	US
Kind code	A1
Filing date	Sep 29, 2023
Priority date	Sep 30, 2022
Publication date	May 16, 2024
Grant date	—

Abstract

Methods and neural network accelerator for online selection of number formats for network parameters of a neural network. The neural network accelerator comprises at least one network processing hardware unit configured to receive network parameters for layers of the neural network and perform one or more neural network operations on the received network parameters in accordance with the neural network; a statistics collection hardware unit configured to collect one or more statistics on a first set of network parameters for a layer while the neural network accelerator is performing a pass of the neural network; and a format conversion hardware unit configured to convert a second set of network parameters to a number format selected based on the collected one or more statistics, the second set of network parameters comprising (i) the first set of network parameters and/or another set of network parameters for the layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters.

First claim

What is claimed is:

1 . A method of dynamically selecting a number format for a set of network parameters of a neural network, the method comprising:
collecting, using a statistics collection hardware unit of a neural network accelerator, one or more statistics on a first set of network parameters for a layer of the neural network while the neural network accelerator is performing a pass of the neural network; selecting a number format based on the collected one or more statistics; converting, using a format conversion hardware unit of the neural network accelerator, a second set of network parameters to the selected number format, the second set of network parameters comprising (i) the first set of network parameters and/or another set of network parameters for the layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters; and processing, using one or more network processing hardware units of the neural network accelerator, the converted second set of network parameters in accordance with the neural network to perform the pass of the neural network or to perform the subsequent pass of the neural network.

2 . A neural network accelerator comprising:
at least one network processing hardware unit configured to receive network parameters for layers of a neural network and perform one or more neural network operations on the received network parameters in accordance with the neural network; a statistics collection hardware unit configured to collect one or more statistics on a first set of network parameters for a layer while the neural network accelerator is performing a pass of the neural network; and a format conversion hardware unit configured to convert a second set of network parameters to a number format selected based on the collected one or more statistics, the second set of network parameters comprising (i) the first set of network parameters and/or another set of network parameters for the layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters.

3 . The neural network accelerator of claim 2 , wherein the neural network accelerator is configured to provide the collected one or more statistics to an external unit coupled to the neural network accelerator, the external unit configured to select the number format based on the collected one or more statistics in accordance with a format selection algorithm.

4 . The neural network accelerator of claim 2 , further comprising a format selection hardware unit that is configured to select the number format based on the collected one or more statistics in accordance with a format select algorithm.

5 . The neural network accelerator of claim 2 , wherein the first set of network parameters comprises all network parameters of a same type for the layer, and the second set of network parameters comprises a set of network parameters for a subsequent pass of the neural network that correspond to the first set of network parameters.

6 . The neural network accelerator of claim 2 , wherein the neural network accelerator is configured to perform a pass of the neural network in a plurality of hardware passes of the neural network accelerator, wherein for each hardware pass the neural network accelerator receives a set of input data corresponding to all or a portion of the input data to a layer of the neural network and processes that set of input data in accordance with at least the layer of the neural network for the pass of the neural network.

7 . The neural network accelerator of claim 6 , wherein the first set of network parameters comprises all of the network parameters of a particular type for a layer that are in a hardware pass of the neural network accelerator for the pass of the neural network.

8 . The neural network accelerator of claim 7 , wherein the second set of network parameters comprises all of the network parameters of the particular type for the layer that are in another hardware pass of the neural network accelerator for the pass of the neural network and/or the first set of network parameters.

9 . The neural network accelerator of claim 7 , wherein the second set of network parameters comprises the network parameters in a hardware pass of the neural network accelerator for a subsequent pass of the neural network that correspond to the first set of network parameters.

10 . The neural network accelerator of claim 6 , wherein the first set of network parameters comprises a subset of the network parameters of a particular type for the layer that are in a hardware pass of the neural network accelerator for the pass of the neural network.

11 . The neural network accelerator of claim 10 , wherein the second set of network parameters comprises another subset of the network parameters of the particular type for the layer that are in the hardware pass of the neural network accelerator for the pass of the neural network and/or the first set of network parameters.

12 . The neural network accelerator of claim 10 , wherein the second set of network parameters comprises the network parameters in a hardware pass of the neural network accelerator for a subsequent pass of the neural network that correspond to the first set of network parameters.

13 . The neural network accelerator of claim 2 , wherein:
the format conversion hardware unit is configured to convert the first set of network parameters to a configurable number format prior to the first set of network parameters being processed by one or more of the at least one network processing hardware unit; and the statistics collection hardware unit is configured to collect the statistics on the first set of network parameters prior to the format conversion performed by the format conversion hardware unit.

14 . The neural network accelerator of claim 2 , wherein:
the format conversion hardware unit is configured to convert the first set of network parameters to a configurable number format prior to the first set of network parameters being processed by one or more of the at least one network processing hardware unit; the statistics collection hardware unit is configured to collect the statistics on the first set of network parameters after the format conversion performed by the format conversion hardware unit; and the second set of network parameters comprises (i) another set of network parameters for the layer for the pass of the neural network, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the first set of network parameters.

15 . The neural network accelerator of claim 2 , further comprising:
another statistics collection hardware unit configured to collect one or more statistics on a third set of network parameters for another layer of the neural network while the neural network accelerator is performing the pass of the neural network; and another format conversion hardware unit configured to convert a fourth set of network parameters to a number format selected based on the one or more statistics collected by the other statistics collection hardware unit, the fourth set of network parameters comprising (i) the third set of network parameters and/or another set of network parameters for the other layer, or (ii) a set of network parameters for a subsequent pass of the neural network corresponding to the third set of network parameters.

16 . The neural network accelerator of claim 2 , wherein the pass of the neural network and the subsequent pass of the neural

Assignees

• Imagination Tech Ltd

Inventors

• Imber James
• Csefalvay Szabolcs
• Choudhury Biswarup
• Dikici Cagatay
• Atherton Timothy

Classifications

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/0464 Primary

  Convolutional networks [CNN, ConvNet] · CPC title