Weight loading in an array

Patent metadata

Field	Value
Publication number	US-11275997-B1
Application number	US-201815967318-A
Country	US
Kind code	B1
Filing date	Apr 30, 2018
Priority date	Apr 30, 2018
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

Disclosed herein are techniques for obtain weights for neural network computations. In one embodiment, an integrated circuit may include memory configured to store a first weight and a second weight; a row of processing elements comprising a first processing element and a second processing element, the first processing element comprising a first weight register, the second processing element comprising a second weight register, both of the first weight register and the second weight register being controllable by a weight load signal; and a controller configured to: provide the first weight from the memory to the row of processing elements; set the weight load signal to enable the first weight to propagate through the row to reach the first processing element; and set the weight load signal to store the first weight at the first weight register and the flush value at the second weight register.

First claim

What is claimed is:

1. A method comprising:
obtaining, from a state buffer of a neural network processor, a first set of weights for neural network computations, the neural network processor comprising a row of processing elements, the row of processing elements comprising first, second, and third processing elements, the third processing element being not involved in the neural network computations;
storing, at the first processing element, a first weight of the first set of weights;
propagating the first weight from the first processing element to the second processing element;
storing a second weight of the first set of weights at the first processing element;
obtaining, from the state buffer, a second set of weights for neural network computations;
storing, at the first processing element, a third weight of the second set of weights;
propagating the third weight from the first processing element to the second processing element;
storing a fourth weight of the second set of weights at the first processing element; and
propagating a flush value from the second processing element to the third processing element instead of propagating the first or second weights to the third processing element.

2. The method of claim 1 , wherein each of the first processing element, the second processing element, and the third processing element includes a pipeline register and a load register, the pipeline register being controllable to select, based on a weight load signal, between the flush value and a weight value for storage at the pipeline register, and the load register being controllable based on the weight load signal to store a weight value;
wherein the method further comprises:
de-asserting the weight load signal to control the pipeline register of the first processing element to store the first weight and to propagate the first weight to the load register of the second processing element;
asserting the weight load signal to:
store a flush value at the pipline registers of the first processing element, the second processing element, and the third processing element,
store the second weight at the load register of the first processing element;
store the first weight at the load register of the second processing element; and
store the flush value at the load register of the third processing element.

3. The method of claim 2 , wherein the method further comprises:
de-asserting the weight load signal to control the pipeline register of the first processing element to store the third weight and to propagate the third weight to the load register of the second processing element;
asserting the weight load signal to:
store the fourth weight at the load register of the first processing element;
store the third weight at the load register of the second processing element; and
maintain the flush value at the load register of the third processing element.

4. An integrated circuit comprising:
memory configured to store a first weight and a second weight; and
a row of processing elements comprising a first processing element and a second processing element;
a controller configured to:
obtain the first weight from the memory;
provide the first weight to the row of processing elements;
propagate the first weight through at least a part of the row of processing elements to reach the first processing element;
store the first weight at a first weight register of the first processing element and a flush value at a second weight register of the second processing element;
obtain the second weight from the memory;
provide the second weight to the row of processing elements;
propagate the second weight through the at least a part of the row of processing elements to reach the first processing element;
propagate the flush value through at least a part of the row of processing elements; and
store the second weight at the first weight register and the flush value at the second weight register.

5. The integrated circuit of claim 4 , wherein each processing element of the row of processing elements includes a weight register and pipeline register;
wherein the pipeline register is controllable to store one of a weight value or the flush value based on a weight load signal;
wherein the weight register is controllable to store a value at an input based on the weight load signal;
wherein an input of the weight register of the first processing element and an input of the pipeline register of the first processing element are coupled with an output of the pipeline register of another weight register;
wherein an input of the weight register of the second processing element and an input of the pipeline register of the second processing element are coupled with an output of the pipeline register of the first processing element;
wherein each pipeline register includes a multiplexer circuit controllable by the weight load signal to provide one of the flush value or the weight value to the each pipeline register for storage;
wherein, when the weight load signal is set to the first weight, the pipeline registers are controlled to store and propagate the weight value; and
wherein, when the weight load signal is set to the second weight:
the pipeline registers are controlled to store and propagate the flush value,
the weight register of the first processing element is controlled to store the first weight, and
the weight register of the second processing element is controlled to store the flush value propagated by the pipeline register of the first processing element.

6. An apparatus, comprising:
a controller; and
a first processing element and a second processing element of a row of a systolic array;
wherein the first processing element includes:
a first register configured to select, based on a control signal from the controller, between a weight value received from the controller and a flush value so as to cause either the weight value or the flush value to be propagated from the first processing element to the second processing element;
a second register configured to store the weight value based on the control signal; and
arithmetic circuit configured to perform arithmetic operations for a neural network based on the weight value provided from the second register.

7. The apparatus of claim 6 , wherein the controller is configured to set the control signal to a first value to control the second register to store the weight value and to control the first register to propagate the flush value to the second processing element.

8. The apparatus of claim 7 , wherein the second processing element further comprises a third register configured to store the weight value or the flush value propagated from the first register based on the control signal;
wherein the controller is configured to:
at a first clock cycle, responsive to the control signal being set to the first value:
store the weight value at the second register; and
store the flush value at the third register of the second processing element; and

at a second clock cycle, responsive to the control signal being set to a second value:
maintain the weight value at the second register; and
maintain the flush value at the third register of the second processing element.

9. The apparatus of claim 8 , wherein the weight value is a first weight value;
wherein the controller is configured to:
at a third clock cycle, responsive to the control signal being set to the second value:
store a second weight value at the first register; and
maintain the flush value at the third register of the second processing element.

10. The apparatus of claim 9 , wherein the controller is further configured to maintain the first weight value

Assignees

• Amazon Tech Inc

Inventors

• Vantrease Dana Michelle
• Diamant Ron
• Amirineni Sundeep

Classifications

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/048

  Activation functions · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title