Method and device with neural network implementation

Patent metadata

Field	Value
Publication number	US-12333418-B2
Application number	US-202318489209-A
Country	US
Kind code	B2
Filing date	Oct 18, 2023
Priority date	Dec 9, 2019
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

A neural network device including an on-chip buffer memory that stores an input feature map of a first layer of a neural network, a computational circuit that receives the input feature map of the first layer through a single port of the on-chip buffer memory and performs a neural network operation on the input feature map of the first layer to output an output feature map of the first layer corresponding to the input feature map of the first layer, and a controller that transmits the output feature map of the first layer to the on-chip buffer memory through the single port to store the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory.

First claim

What is claimed is:

1. A neural network device, comprising:
an on-chip buffer memory configured to store an input feature map of a first layer of a neural network;
a local bus connecting the on-chip buffer memory to a computational circuit;
the computational circuit configured to receive the input feature map of the first layer as read from a single port of the on-chip buffer memory, and as transmitted from the single port to the computational circuit via the local bus, and perform a neural network operation on the input feature map of the first layer to output an output feature map of the first layer corresponding to the input feature map of the first layer; and
a controller configured to control transmission of the output feature map of the first layer to be outputted from the computational circuit via the local bus to the on-chip buffer memory to store the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory,
wherein the output feature map of the first layer is reused, from the on-chip buffer memory, as an input feature map for a neural network operation of a second layer subsequent to the first layer,
wherein, when a read operation of the input feature map of the first layer and a write operation of the output feature map of the first layer are simultaneously requested, an order of the read operation and the write operation are adjusted on the local bus, to prevent a collision between the read operation and the write operation in the single port,
wherein the computational circuit is further configured to perform the neural network operation based on one or more operation loops,
wherein the controller is further configured to perform the read operation of reading data constituting, at least, a portion of the input feature map of the first layer from the on-chip buffer memory through the single port at each cycle in which each of the one or more operation loops is executed, and
when the write operation for writing data constituting, at least, a portion of the output feature map of the first layer to the on-chip buffer memory through the single port is requested at a timing at which the read operation is to be performed, the write operation is performed in preference to the read operation.

2. The neural network device of claim 1 , wherein the read operation is performed whenever a range of features of the input feature map of the first layer is changed, and
wherein the write operation is performed whenever a number of rows or a number of columns in the output feature map of the first layer is changed.

3. The neural network device of claim 1 , wherein the controller
allocates, in different memory address directions of the one memory address space, a first memory address of the on-chip buffer memory for storing the input feature map of the first layer and a second memory address of the on-chip buffer memory for storing the output feature map of the first layer concurrent with the storing of the input feature map at the first memory address,
allocates the first memory address in a first direction from a start point of the one memory address space of the on-chip buffer memory, and
allocates the second memory address in a second direction, opposite to the first direction, from a last point of the one memory space of the on-chip buffer memory.

4. The neural network device of claim 3 , wherein,
when the output feature map of the second layer, corresponding to the input feature map of the second layer, is output from the computational circuit as the output feature map of the first layer stored in the second memory address and reused as the input feature map of the second layer,
the controller allocates a third memory address of the on-chip buffer memory for storing the output feature map of the second layer in the first direction from the start point.

5. The neural network device of claim 1 , wherein the neural network operation includes a convolution operation, an activation operation, and a pooling operation,
wherein the computational circuit is further configured to output, as the output feature map of the first layer, a result of performing the pooling operation and the convolution operation and the activation operation on the input feature map of the first layer.

6. The neural network device of claim 1 , further comprising:
a weight buffer memory configured to store weight values of the first layer of the neural network operation on the input feature map of the first layer,
wherein the weight buffer memory receives the weight values of the first layer from an external memory external to the neural network device through a single port of the weight buffer memory, and transmits the weight of the first layer to the computational circuit through a single port of the weight buffer memory.

7. The neural network device of claim 1 , wherein
the on-chip buffer memory, the computational circuit, and the controller are mounted in a single chip.

8. The neural network device of claim 1 , further comprising:
an auxiliary buffer memory,
wherein, when the output feature map of the second layer, corresponding to the input feature map of the second layer, is output from the computational circuit, the controller determines whether a total size of the input feature map of the second layer and the output feature map of the second layer exceeds a size of the on-chip buffer memory, and when the total size exceeds the size of the on-chip buffer memory, the controller temporarily stores the output feature map of the second layer in the auxiliary buffer memory instead of the on-chip buffer memory, and
wherein the output feature map of the second layer temporarily stored in the auxiliary buffer memory is transmitted to an external memory outside the neural network device based on a preset period.

9. The neural network device of claim 8 , wherein, when an output feature map of a third layer, corresponding to an input feature map of the third layer, is output from the computational circuit as the output feature map of the second layer and reused as an input feature map for a neural network operation of the third layer subsequent to the second layer, the controller determines whether a size of the output feature map of the third layer exceeds the size of the on-chip buffer memory,
wherein, when the size of the output feature map of the third layer exceeds the size of the on-chip buffer memory, the controller temporarily stores the output feature map of the third layer in the auxiliary buffer memory, and
wherein, when the size of the output feature map of the third layer is less than or equal to the size of the on-chip buffer memory, the controller stores the output feature map of the third layer in the on-chip buffer memory.

10. A processor-implemented method, the method comprising:
storing an input feature map of a first layer of a neural network in an on-chip buffer memory;
transmitting the input feature map of the first layer from a single port of the on-chip buffer memory to a computational circuit via a local bus connecting the computational circuit to the on-chip buffer memory;
outputting an output feature map of the first layer, corresponding to the input feature map of the first layer, upon the computational circuit performing a neural network operation on the input feature map of the first layer;
transmitting the output feature map of the first layer to the on-chip buffer memory via the local bus and through the single port, thereby storing the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory;
performing a read operation through the single port of reading data constituti

Assignees

• Samsung Electronics Co Ltd
• Ulsan Nat Inst Science & Tech Unist

Inventors

• Yu Hyeongseok
• Sim Hyeonuk
• Lee Jongeun

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title