Hardware implementation of a deep neural network with variable output data format

Patent metadata

Field	Value
Publication number	US-12165045-B2
Application number	US-201816136553-A
Country	US
Kind code	B2
Filing date	Sep 20, 2018
Priority date	Sep 20, 2017
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

Hardware implementations of DNNs and related methods with a variable output data format. Specifically, in the hardware implementations and methods described herein the hardware implementation is configured to perform one or more hardware passes to implement a DNN wherein during each hardware pass the hardware implementation receives input data for a particular layer, processes that input data in accordance with the particular layer (and optionally one or more subsequent layers), and outputs the processed data in a desired format based on the layer, or layers, that are processed in the particular hardware pass. In particular, when a hardware implementation receives input data to be processed, the hardware implementation also receives information indicating the desired format for the output data of the hardware pass and the hardware implementation is configured to, prior to outputting the processed data convert the output data to the desired format.

First claim

What is claimed is:

1. A method in a hardware implementation of a Deep Neural Network (DNN) configured to implement the DNN by processing data using one or more hardware passes, wherein during each hardware pass the hardware implementation receives at least a portion of input data for a layer of the DNN and processes the received input data in accordance with at least that layer to produce processed data, the method comprising:
receiving, at an input module of the hardware implementation, a set of input data for a hardware pass of the hardware implementation, the set of input data representing at least a portion of input data for a particular layer of the DNN;
receiving, at a decoder of the hardware implementation, information indicating a desired output data format for the hardware pass;
processing, at a processing module of the hardware implementation, the set of input data according to one or more layers of the DNN associated with the hardware pass to produce processed data, the one or more layers comprising the particular layer of the DNN, the processing module comprising a plurality of sub-processing modules, each sub-processing module of the plurality of sub-processing modules configured to perform all or part of the processing of one type of layer, at least one of the plurality of sub-processing modules comprising a format conversion module configured to receive information indicating a format of input data to the corresponding sub-processing module and receive information indicating a desired internal data format for the hardware pass, the desired internal data format for the hardware pass being independent from the desired output data format for the hardware pass, wherein processing the set of input data according to the one or more layers of the DNN associated with the hardware pass comprises generating an internal data set and converting, using the format conversion module of one of the at least one sub-processing module, the internal data set from the format of the input data to the sub-processing module into the desired internal data format for the hardware pass prior to processing that internal data set at the one of the at least one sub-processing module, the internal data set being data within the hardware implementation; and
converting, at an output module of the hardware implementation, the processed data into the desired output data format for the hardware pass to produce output data for the hardware pass.

2. A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform the method as set forth in claim 1 .

3. A hardware implementation of a Deep Neural Network (DNN) configured to implement the DNN by processing data using one or more hardware passes, wherein during each hardware pass the hardware implementation receives at least a portion of input data for a layer of the DNN and processes the received input data in accordance with at least that layer to produce processed data, the hardware implementation comprising:
an input module configured to receive a set of input data for a hardware pass of the hardware implementation, the set of input data representing at least a portion of input data for a particular layer of the DNN;
a decoder configured to receive information indicating a desired output data format for the hardware pass;
a processing module configured to process the set of input data according to one or more layers of the DNN associated with the hardware pass to produce processed data, the one or more layers comprising the particular layer of the DNN, the processing module comprising a plurality of sub-processing modules, each sub-processing module of the plurality of sub-processing modules configured to perform all or part of the processing of one type of layer, at least one of the plurality of sub-processing modules comprising a format conversion module configured to receive information indicating a format of input data to the corresponding sub-processing module and receive information indicating a desired internal data format for the hardware pass, the desired internal data format for the hardware pass being independent from the desired output data format, wherein processing the set of input data according to the one or more layers of the DNN associated with the hardware pass comprises generating an internal data set and converting, using the format conversion module of one of the at least one sub-processing module, the internal data set from the format of the input data to the corresponding sub-processing module into the desired internal data format for the hardware pass prior to processing the internal data set at the one of the at least one sub-processing module, the internal data set being data within the hardware implementation; and
an output module configured to convert the processed data into the desired output data format for the hardware pass to produce output data for the hardware pass.

4. The hardware implementation of claim 3 , wherein:
the input module is further configured to receive a second set of input data for a different hardware pass of the hardware implementation;
the decoder is further configured to receive information indicating a desired output data format for the different hardware pass, the desired output data format for the different hardware pass being different from the desired output data format for the hardware pass;
the processing module is further configured to process the second set of input data according to one or more layers of the DNN associated with the different hardware pass to produce second processed data; and
the output module is further configured to convert the second processed data into the desired output data format for the different hardware pass to produce second output data.

5. The hardware implementation of claim 4 , wherein the second set of input data comprises a portion of input data for the particular layer of the DNN; or wherein the second set of input data comprises at least a portion of input data for another layer of the DNN.

6. The hardware implementation of claim 5 , wherein the second set of input data comprises a least a portion of input data for another layer and the second set of input data comprises at least a portion of the output data in the desired output data format for the hardware pass.

7. The hardware implementation of claim 3 , wherein the output module is further configured to store the output data in the desired output data format for the hardware pass in memory; and the input module is further configured to read the output data in the desired output data format for the hardware pass from memory as a set of input data for another hardware pass.

8. The hardware implementation of claim 3 , wherein the one or more layers of the DNN associated with the hardware pass comprises at least two layers and the processing module is configured to process the set of input data according to the at least two layers by processing the set of input data according to one layer of the at least two layers using a first input data format and processing the set of input data according to another layer of the at least two layers using a second input data format wherein the first input data format and the second input data format are independent from the desired output data format for the hardware pass.

9. The hardware implementation of claim 8 , wherein each of the first and second input data formats is a fixed point format defined by an exponent and an integer bit-width and the exponent of the first data format is different than the exponent of the second data format.

10. The hardware implementation of claim 8 , wherein the

Assignees

• Imagination Tech Ltd

Inventors

• Martin Chris
• Hough David
• Brasnett Paul
• Dikici Cagatay
• Imber James
• Gibson Clifford

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F7/483

  Computations with numbers represented by a non-linear combination of denominational numbers, e.g. rational numbers, logarithmic number system or floating-point numbers {(G06F7/4806, G06F7/4824, G06F7/49, G06F7/491, G06F7/544 take precedence)} · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06F1/3275 Primary

  Power saving in memory, e.g. RAM, cache · CPC title

• G06F2207/4824

  Neural networks · CPC title