Vector computation unit in a neural network processor

Patent metadata

Field	Value
Publication number	US-11620508-B2
Application number	US-201916245406-A
Country	US
Kind code	B2
Filing date	Jan 11, 2019
Priority date	May 21, 2015
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.

First claim

What is claimed is:

1. A circuit for performing neural network computations, the circuit comprising:
a vector computation unit, wherein the vector computation unit includes:
pooling circuitry, the pooling circuitry being configured to:
receive, from other circuitry included in the vector computation unit, a set of values that are computational outputs of a neural network layer;
perform a particular pooling function on the set of values, the particular pooling function being used to pool one or more values in the set of values; and
generate a pooled value based on the particular pooling function that is used to pool the one or more values in the set of values; and

a plurality of registers and a plurality of memory units for storing the set of values, each register being connected in series to a respective memory unit, wherein each register is configured to store a value of the set of values and each memory unit is configured to store a plurality of values of the set of values.

2. The circuit of claim 1 , wherein the set of values comprises normalized values and the pooling circuitry comprises aggregation circuitry configured to:
apply an aggregation function to one or more of the normalized values to generate the pooled value, the aggregation function representing the particular pooling function that is used to pool the one or more values in the set of values.

3. The circuit of claim 2 , wherein the aggregation function is operable to cause the pooling circuitry to return:
a maximum, a minimum, or an average of the normalized values in the set of values; or
a maximum, a minimum, or an average of a subset of the normalized values in the set of values.

4. The circuit of claim 1 , wherein the particular pooling function is specified by a control signal received by the vector computation unit.

5. The circuit of claim 1 , wherein the pooling circuitry is further configured to:
perform a pooling of an M×N set of values based on the particular pooling function, where M and N are respective integers that are each greater than or equal to one.

6. The circuit of claim 1 , wherein:
the vector computation unit includes multiple parallel pooling circuitries; and
each pooling circuitry of the multiple parallel pooling circuitries is configured receive, over a given clock cycle, a respective element from the set of values.

7. The circuit of claim 1 , wherein the pooling circuitry is configured to, after every clock cycle, shift a given value in the set of values to a subsequent register or memory unit of the pooling circuitry to generate the pooled value from the set of values.

8. The circuit of claim 1 , wherein the set of values comprises a vector of accumulated values and the pooling circuitry generates the pooled value based on a particular pooling function that is specified by a control signal received by the vector computation unit.

9. The circuit of claim 8 , wherein the control signal specifies one or more parameters that are used by the pooling circuitry to pool the one or more values in the set of values, at least one parameter comprising a stride value for a particular neural network layer of the neural network.

10. A method for performing neural network computations, the method comprising:
receiving, by a pooling circuitry included in a vector computation unit, a set of values for performing the neural network computations;
performing, by the pooling circuitry, a particular pooling function on the set of values, the particular pooling function being used to pool one or more values in the set of values;
generating, by the pooling circuitry, a pooled value based on the particular pooling function that is used to pool the one or more values in the set of values;
providing, by the pooling circuitry, the pooled value to other circuitry included in the vector computation unit to perform the neural network computations; and
storing the set of values in a plurality of registers and a plurality of memory units, each register being connected in series to a respective memory unit, wherein each register is configured to store a value of the set of values and each memory unit is configured to store a plurality of values of the set of values.

11. The method of claim 10 , wherein the set of values comprises normalized values and the method further comprises:
applying, by the pooling circuitry, an aggregation function to one or more of the normalized values to generate the pooled value, the aggregation function representing the particular pooling function that is used to pool the one or more values in the set of values.

12. The method of claim 11 , wherein the aggregation function is operable to cause the pooling circuitry to return:
a maximum, a minimum, or an average of the normalized values in the set of values; or
a maximum, a minimum, or an average of a subset of the normalized values in the set of values.

13. The method of claim 12 , wherein the particular pooling function is specified by a control signal received by the vector computation unit.

14. The method of claim 10 , further comprising:
shifting, by the pooling circuitry and after every clock cycle, a given value in the set of values to a subsequent register or memory unit of the pooling circuitry to generate the pooled value from the set of values.

15. The method of claim 10 , wherein generating the pooled value comprises:
generating the pooled value based on a particular pooling function that is specified by a control signal received by the vector computation unit.

16. The method of claim 15 , further comprising:
pooling, by the pooling circuitry, the one or more values in the set of values based on a control signal that specifies one or more parameters for pooling the one or more values, at least one parameter comprising a stride value for a particular neural network layer of the neural network.

17. The method of claim 10 , wherein performing the particular pooling function on the set of values comprises:
performing a pooling of an M×N set of values based on the particular pooling function, where M and N are respective integers that are each greater than or equal to one.

18. A non-transitory machine-readable storage device for storing instructions executable by a processor to perform neural network computations, the instructions comprising:
receiving, by a pooling circuitry included in a vector computation unit, a set of values for performing the neural network computations;
performing, by the pooling circuitry, a particular pooling function on the set of values, the particular pooling function being used to pool one or more values in the set of values;
generating, by the pooling circuitry, a pooled value based on the particular pooling function that is used to pool the one or more values in the set of values;
providing, by the pooling circuitry, the pooled value to other circuitry included in the vector computation unit to perform the neural network computations; and
storing the set of values in a plurality of registers and a plurality of memory units, each register being connected in series to a respective memory unit, wherein each register is configured to store a value of the set of values and each memory unit is configured to store a plurality of values of the set of values.

Assignees

• Google Llc

Inventors

• Thorson Gregory Michael
• Clark Christopher Aaron
• Luu Dan

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N3/048

  Activation functions · CPC title