Runtime reconfigurable neural network processor core

Patent metadata

Field	Value
Publication number	US-12056598-B2
Application number	US-202218046301-A
Country	US
Kind code	B2
Filing date	Oct 13, 2022
Priority date	Jun 19, 2018
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

Hardware neural network processors, are provided. A neural core includes a weight memory, an activation memory, a vector-matrix multiplier, and a vector processor. The vector-matrix multiplier is adapted to receive a weight matrix from the weight memory, receive an activation vector from the activation memory, and compute a vector-matrix multiplication of the weight matrix and the activation vector. The vector processor is adapted to receive one or more input vector from one or more vector source and perform one or more vector functions on the one or more input vector to yield an output vector. In some embodiments a programmable controller is adapted to configure and operate the neural core.

First claim

What is claimed is:

1. A neural core device comprising:
a weight memory;
an activation memory;
a vector-matrix multiplier adapted to receive a weight matrix from the weight memory, receive an activation vector from the activation memory, compute a set of partial products by vector-matrix multiplication of the weight matrix and the activation vector, and output the set of partial products; and
a vector processor adapted to receive one or more input vector including the set of partial products from one or more vector source including the vector matrix multiplier and perform one or more vector functions on the one or more input vector to yield an output vector.

2. The neural core device of claim 1 , further comprising a microengine device operatively coupled to the vector processor.

3. The neural core device of claim 2 , wherein the microengine device is adapted to:
map the one or more vector source to the vector processor,
map the vector processor to one or more vector targets,
instruct the vector processor to perform the one or more vector functions and provide the output vector to the one or more vector targets.

4. The neural core device of claim 1 , further comprising:
an activation unit operatively coupled to the vector processor and adapted to:
apply an activation function to the results from the vector processor.

5. The neural core device of claim 4 , wherein the activation function is configurable.

6. The neural core device of claim 5 , wherein:
the microengine device is further adapted to instruct the activation unit to compute the activation function and provide results to the activation memory.

7. The neural core device of claim 1 , wherein the one or more vector sources comprise a partial sum memory, a network, a register, or a parameter memory.

8. The neural core device of claim 1 , wherein the vector targets comprise an activation memory, a partial sum memory, a register, or a network.

9. The neural core device of claim 1 , wherein the vector processor is adapted to apply one or more constant to the results.

10. The neural core device of claim 1 , configured to accumulate partial sums.

11. The neural core device of claim 1 , wherein:
the microengine device is further adapted to instruct the vector-matrix multiplier to read a weight matrix from the weight memory, read an activation vector from the activation memory, and to compute a vector-matrix multiplication of the weight matrix and the activation vector.

12. The neural core device of claim 11 , wherein:
the weight matrix is a subarray of a neural network weight matrix;
the activation vector is a subarray of a neural network activation vector.

13. The neural core device of claim 12 , wherein the vector-matrix multiplication of the weight matrix and the activation vector is provided for accumulation, said accumulation yielding a vector-matrix multiplication of the neural network weight matrix and the neural network activation vector.

14. The neural core device of claim 1 , configured to compute a neural network function having an input, parameters, and an output.

15. The neural core device of claim 14 , wherein the weight matrix and/or the activation vector have configurable sizes.

16. The neural core device of claim 14 , wherein the neural network input, parameters, and/or output have configurable sizes.

17. The neural core device of claim 14 , wherein the neural network function is configurable.

18. The neural core device of claim 1 , configured to compute a neural network function in conjunction with a plurality of additional neural cores interconnected by a network.

19. The neural core device of claim 1 , configured to compute a portion of a neural network function.

20. The neural core device of claim 19 , wherein the portion of the neural network function is configurable.

21. A method comprising:
receiving a weight matrix from a weight memory;
receiving an activation vector from an activation memory;
computing a set of partial products by vector-matrix multiplication of the weight matrix and the activation vector;
performing one or more vector functions on the set of partial products to yield an output vector;
applying an activation function to the output vector to determine a result; and
providing the result to the activation memory.

22. The method of claim 21 , wherein the weight matrix is a subarray of a neural network weight matrix and the activation vector is a subarray of a neural network activation vector, the method further comprising:
accumulating the result with additional results to yield a vector-matrix multiplication of the neural network weight matrix and the neural network activation vector.

23. The method of claim 21 , further comprising simultaneously storing the activation vector and the result in the activation memory.

24. A method comprising:
identifying one or more of a plurality of vector sources for a vector processor;
identifying one or more of a plurality of vector targets for the vector processor; and
performing by the vector processor a vector function on input including a set of partial products from the one or more of the plurality of sources and provide results to the one or more of the plurality of vector targets.

Assignees

• IBM

Inventors

• Cassidy Andrew S
• Appuswamy Rathinakumar
• Arthur John V
• Datta Pallab
• Esser Steven K
• Flickner Myron D
• Klamo Jennifer
• Modha Dharmendra S
• Penner Hartmut
• Sawada Jun
• Taba Brian

Classifications

• G06N3/048 Primary

  Activation functions · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G11C11/54

  using elements simulating biological cells, e.g. neuron · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06N3/063 Primary

  using electronic means · CPC title