Method and device with neural network implementation

Patent metadata

Field	Value
Publication number	US-11829862-B2
Application number	US-202016893560-A
Country	US
Kind code	B2
Filing date	Jun 5, 2020
Priority date	Dec 9, 2019
Publication date	Nov 28, 2023
Grant date	Nov 28, 2023

Abstract

A neural network device includes: an on-chip buffer memory that stores an input feature map of a first layer of a neural network, a computational circuit that receives the input feature map of the first layer through a single port of the on-chip buffer memory and performs a neural network operation on the input feature map of the first layer to output an output feature map of the first layer corresponding to the input feature map of the first layer, and a controller that transmits the output feature map of the first layer to the on-chip buffer memory through the single port to store the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory.

First claim

What is claimed is:

1. A neural network device, comprising:
an on-chip buffer memory configured to store an input feature map of a first layer of a neural network;
a computational circuit configured to receive the input feature map of the first layer through a single port of the on-chip buffer memory, and perform a neural network operation on the input feature map of the first layer to output an output feature map of the first layer corresponding to the input feature map of the first layer; and
a controller configured to transmit, through the single port, the output feature map of the first layer to the on-chip buffer memory to store the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory,
wherein the output feature map of the first layer is reused as an input feature map for a neural network operation of a second layer subsequent to the first layer,
wherein the controller is further configured to:
allocate a first memory address in a first direction from a start point of a memory address corresponding to a storage space of the on-chip buffer memory, and
allocate a second memory address in a second direction, opposite to the first direction, from a last point of the memory address corresponding to the storage space of the on-chip buffer memory.

2. The neural network device of claim 1 , wherein the computational circuit is further configured to perform the neural network operation based on one or more operation loops,
wherein the controller is further configured to perform a read operation of reading data constituting, at least, a portion of the input feature map of the first layer from the on-chip buffer memory through the single port at each cycle in which each of the one or more operation loops is executed, and
when a write operation for writing data constituting, at least, a portion of the output feature map of the first layer to the on-chip buffer memory through the single port is requested at a timing at which the read operation is to be performed, the write operation is performed in preference to the read operation.

3. The neural network device of claim 1 , wherein,
when the output feature map of the second layer, corresponding to the input feature map of the second layer, is output from the computational circuit as the output feature map of the first layer stored in the second memory address and reused as the input feature map of the second layer,
the controller allocates a third memory address of the on-chip buffer memory for storing the output feature map of the second layer in the first direction from the start point.

4. The neural network device of claim 1 , wherein the neural network operation includes a convolution operation, an activation operation, and a pooling operation,
wherein the computational circuit is further configured to output, as the output feature map of the first layer, a result of performing the pooling operation and the convolution operation and the activation operation on the input feature map of the first layer.

5. The neural network device of claim 1 , further comprising:
a weight buffer memory configured to store weight values of the first layer of the neural network operation on the input feature map of the first layer,
wherein the weight buffer memory receives the weight values of the first layer from an external memory external to the neural network device through a single port of the weight buffer memory, and transmits the weight of the first layer to the computational circuit through a single port of the weight buffer memory.

6. The neural network device of claim 1 , wherein
the on-chip buffer memory, the computational circuit, and the controller are mounted in a single chip.

7. The neural network device of claim 1 , further comprising:
an auxiliary buffer memory,
wherein, when the output feature map of the second layer, corresponding to the input feature map of the second layer, is output from the computational circuit, the controller determines whether a total size of the input feature map of the second layer and the output feature map of the second layer exceeds a size of the on-chip buffer memory, and when the total size exceeds the size of the on-chip buffer memory, the controller temporarily stores the output feature map of the second layer in the auxiliary buffer memory instead of the on-chip buffer memory, and
wherein the output feature map of the second layer temporarily stored in the auxiliary buffer memory is transmitted to an external memory outside the neural network device based on a preset period.

8. The neural network device of claim 7 , wherein,
when an output feature map of a third layer, corresponding to an input feature map of the third layer, is output from the computational circuit as the output feature map of the second layer and reused as an input feature map for a neural network operation of the third layer subsequent to the second layer, the controller determines whether a size of the output feature map of the third layer exceeds the size of the on-chip buffer memory, and
when the size of the output feature map of the third layer exceeds the size of the on-chip buffer memory, the controller temporarily stores the output feature map of the third layer in the auxiliary buffer memory, and
when the size of the output feature map of the third layer is less than or equal to the size of the on-chip buffer memory, the controller stores the output feature map of the third layer in the on-chip buffer memory.

9. A processor-implemented method, the method comprising:
storing an input feature map of a first layer of a neural network in an on-chip buffer memory;
transmitting the input feature map of the first layer to a computational circuit through a single port of the on-chip buffer memory;
outputting an output feature map of the first layer, corresponding to the input feature map of the first layer, upon the computational circuit performing a neural network operation on the input feature map of the first layer; and
transmitting the output feature map of the first layer to the on-chip buffer memory through the single port, thereby storing the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory,
wherein the output feature map of the first layer is reused as an input feature map for a neural network operation of a second layer subsequent to the first layer, and
wherein the storing of the output feature map of the first layer and the input feature map of the first layer together in the on-chip buffer memory comprises:
allocating, in a first direction from a start point of a memory address corresponding to a storage space of the on-chip buffer memory, a first memory address for storing the input feature map of the first layer, and
allocating, in a second direction opposite to the first direction from a last point of the memory address corresponding to the storage space of the on-chip buffer memory, the second memory address for storing the output feature map of the first layer.

10. The method of claim 9 , further comprising:
performing a read operation through the single port of reading data constituting, at least, a portion of the input feature map of the first layer from the on-chip buffer memory at each cycle in which each of one or more operation loops is executed, in order to perform the neural network operation based on the one or more operation loops; and
when a write operation for writing data constituting, at least, a portion of the output feature map of the first layer to the on-chip buffer memory through the single port is requested at a timing at which the read operation is to be per

Assignees

• Samsung Electronics Co Ltd
• Ulsan Nat Inst Science & Tech Unist

Inventors

• Yu Hyeongseok
• Sim Hyeonuk
• Lee Jongeun

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title