Neural network arithmetic processing device and neural network arithmetic processing method

Patent metadata

Field	Value
Publication number	US-11681498-B2
Application number	US-202016819303-A
Country	US
Kind code	B2
Filing date	Mar 16, 2020
Priority date	Mar 27, 2019
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

A neural network arithmetic processing device is capable of implementing a further increase in speed and efficiency of multiply-accumulate arithmetic operation, suppressing an increase in circuit scale, and performing multiply-accumulate arithmetic operation with simple design. A neural network arithmetic processing device includes a first multiply-accumulate arithmetic unit, a register connected to the first multiply-accumulate arithmetic unit, and a second multiply-accumulate arithmetic unit connected to the register. The first multiply-accumulate arithmetic unit has a first memory, a second memory, a first multiplier, a first adder, and a first output unit. The second multiply-accumulate arithmetic unit has an input unit, a third memory, second multipliers, second adders, and second output units.

First claim

The invention claimed is:

1. A neural network arithmetic processing device comprising:
at least one first multiply-accumulate arithmetic unit;
at least one register connected to the at least one first multiply-accumulate arithmetic unit; and
at least one second multiply-accumulate arithmetic unit connected to the at least one register,
wherein each of the at least one first multiply-accumulate arithmetic units has
a first memory that stores a plurality of first input variables,
a second memory that stores a plurality of pieces of first weight data,
at least one first multiplier that performs at least one multiply operation that calculates a plurality of products of the first input variables and the first weight data,
at least one first adder that performs at least one add operation that calculates a plurality of sums of the products multiplied by the at least one first multiplier, and
at least one first output unit that outputs the plurality of the sums added by the at least one first adder to the at least one register as a plurality of second input variables, and

each of the at least one second multiply-accumulate arithmetic units has
a third memory that stores a plurality of pieces of second weight data,
at least one second multiplier that performs at least one multiply operation that calculates a plurality of products of the second weight data and the second input variables held in the at least one register,
at least one second adder that performs at least one add operation that calculates a plurality of sums of the products multiplied by the at least one second multiplier, and
at least one second output unit that outputs the plurality of sums added by the at least one second adder as a plurality of output values,

wherein each of the second memory and the third memory is a ring buffer memory.

2. The neural network arithmetic processing device according to claim 1 ,
wherein at least part of the multiply and add operations to be executed by the second multiply-accumulate arithmetic units is executed in parallel with the multiply and add operations to be executed by the first multiply-accumulate arithmetic units.

3. The neural network arithmetic processing device according to claim 2 ,
wherein, in a case where a number of the multiply and add operations constituting arithmetic processing P 1 to be executed by the first multiply-accumulate arithmetic unit 10 is (n+1) (where n is an integer equal to or greater than 0), and a number of the multiply and add operations constituting arithmetic processing P 2 to be executed by the second multiply-accumulate arithmetic unit is (q+1) (where q is an integer equal to or greater than 0),
a number of parallel multiply and add operations L 1 of the arithmetic processing P 1 to be executed by the first multiply-accumulate arithmetic unit is a divisor of the number of arithmetic operations (n+1), and a number of parallel multiply and add operations L 2 of the arithmetic processing P 2 to be executed by the second multiply-accumulate arithmetic unit is a divisor of the number of arithmetic operations (q+1).

4. The neural network arithmetic processing device according to claim 3 ,
wherein both of a number of pipelines C 1 constituting the arithmetic processing P 1 and a number of pipelines C 2 constituting the arithmetic processing P 2 are a common divisor of the number of arithmetic operations (n+1) and the number of arithmetic operations (q+1).

5. The neural network arithmetic processing device according to claim 4 ,
wherein both of the number of pipelines C 1 constituting the arithmetic processing P 1 and the number of pipelines C 2 constituting the arithmetic processing P 2 are a greatest common divisor of the number of arithmetic operations (n+1) and the number of arithmetic operations (q+1).

6. The neural network arithmetic processing device according to claim 3 ,
wherein a timing at which the arithmetic processing to be executed by the second multiply-accumulate arithmetic unit ends is the same as a timing at which the arithmetic processing to be executed by the first multiply-accumulate arithmetic unit ends.

7. The neural network arithmetic processing device according to claim 2 ,
wherein the at least one first multiplier is a plurality of first multipliers, and
the at least one first adder is a plurality of first adders.

8. The neural network arithmetic processing device according to claim 2 ,
wherein the at least one second multiply-accumulate arithmetic unit is a plurality of the second multiply-accumulate arithmetic units, and
the plurality of the second multiply-accumulate arithmetic units operate in parallel with the at least one first multiply-accumulate arithmetic unit.

9. The neural network arithmetic processing device according to claim 2 ,
wherein the at least one first multiply-accumulate arithmetic unit is a plurality of the first multiply-accumulate arithmetic units, and
the plurality of the first multiply-accumulate arithmetic units operate in parallel with the at least one second multiply-accumulate arithmetic unit.

10. The neural network arithmetic processing device according to claim 2 ,
wherein the at least one first multiply-accumulate arithmetic unit is a plurality of the first multiply-accumulate arithmetic units, and
the at least one second multiply-accumulate arithmetic unit is a plurality of the second multiply-accumulate arithmetic units, and
the plurality of the first multiply-accumulate arithmetic units operate in parallel with the plurality of the second multiply-accumulate arithmetic units.

11. The neural network arithmetic processing device according to claim 1 , further comprising:
at least one activation function arithmetic processing unit that is provided at the at least one first multiply-accumulate arithmetic unit and/or and the at least one second multiply-accumulate arithmetic unit and is located at least either between the at least one first multiply-accumulate arithmetic unit and the register or between the register and the at least one second multiply-accumulate arithmetic unit to perform an arithmetic operation using an activation function.

12. The neural network arithmetic processing deVice according to claim 1 ,
wherein the at least one first multiply-accumulate arithmetic unit is a plurality of the first multiply-accumulate arithmetic units, and
the at least one second multiply-accumulate arithmetic unit is a plurality of the second multiply-accumulate arithmetic units, and
the plurality of the first multiply-accumulate arithmetic units operate in parallel with the plurality of the second multiply-accumulate arithmetic units.

Assignees

• Tdk Corp

Inventors

• SUDA Keita

Classifications

• G06F7/5443 Primary

  Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00) · CPC title

• G06F7/50

  Adding; Subtracting (G06F7/483 - G06F7/491, G06F7/544 - G06F7/556 take precedence) · CPC title

• G06F7/523

  Multiplying only · CPC title

• G06N3/063

  using electronic means · CPC title

• G06F2207/4824

  Neural networks · CPC title