Method and apparatus for controlling memory using prefetch information

Patent metadata

Field	Value
Publication number	US-2021157734-A1
Application number	US-202016953242-A
Country	US
Kind code	A1
Filing date	Nov 19, 2020
Priority date	Nov 27, 2019
Publication date	May 27, 2021
Grant date	—

Abstract

A method for controlling a memory from which data is transferred to a neural network processor and an apparatus thereof are provided, the method including: generating prefetch information of data by using a blob descriptor and a reference prediction table after history information is input; reading the data in the memory based on the pre-fetch information and temporarily archiving read data in a prefetch buffer; and accessing next data in the memory based on the prefetch information and temporarily archiving the next data in the prefetch buffer after the data is transferred to the neural network from the prefetch buffer.

First claim

What is claimed is:

1 . A method for controlling a memory from which data is transferred to a neural network processor, the method comprising:
generating prefetch information of data by using a blob descriptor and a reference prediction table after history information is input; reading the data in the memory based on the pre-fetch information and temporarily archiving read data in a prefetch buffer; and accessing next data in the memory based on the prefetch information and temporarily archiving the next data in the prefetch buffer after the data is transferred to the neural network from the prefetch buffer.

2 . The method of claim 1 , wherein the prefetch information includes first prefetch information about a source address of the data and a size of the data and second prefetch information about a source address of the next data and a size of the next data.

3 . The method of claim 2 , wherein the reading the data in the memory based on the prefetch information and temporarily archiving read data in a prefetch buffer comprises
reading the data in the memory based on the source address in the first prefetch information.

4 . The method of claim 3 , further comprising:
storing, after data updated based on operation using the data is archived in the prefetch buffer, updated data in the memory according to a source address of the data.

5 . An apparatus for controlling a memory from which data is transferred to a neural network processor, the apparatus comprising:
a synapse correlation table configured to include a reference prediction table used for generating prefetch information of data in cooperation with a blob descriptor after history information is input; a low level memory controller configured to read the data in the memory based on the prefetch information and read next data following the data in the memory based on the prefetch information; and a prefetch buffer configured to temporarily store the data until the data read by the low level memory controller is transferred to the neural network processor and temporarily store the next data after the data is transferred to the neural network processor.

6 . The apparatus of claim 5 , wherein the prefetch information includes first prefetch information about a source address of the data and a size of the data and second prefetch information about a source address of the next data and a size of the next data.

7 . The apparatus of claim 6 , wherein the low level memory controller is further configured to read the data in the memory based on the source address of the data included in the first prefetch information and read the next data in the memory based on the source address of the next data included in the second prefetch information.

8 . The apparatus of claim 7 , wherein the prefetch buffer includes a read buffer and a write buffer and the prefetch buffer is configured to temporarily store the data in the read buffer and store the next data in the read buffer after the data is transferred to the neural network processor.

9 . The apparatus of claim 8 , wherein the prefetch buffer is further configured to temporarily store, when data updated based on operation using the data is received from the neural network processor, updated data in the write buffer.

10 . The apparatus of claim 9 , wherein the low level memory controller is further configured to store, when the updated data is temporarily stored in the write buffer, the updated data in the memory according to a target address corresponding to the source address of the data.

11 . A system for operating a neural network, the system comprising:
a neural network processor including a plurality of MAC operator, a memory, and a memory controller configured to transfer data stored in the memory to the neural network processor and store data updated by the neural network processor in the memory, wherein the memory controller is configured to generate prefetch information of data by using a synapse correlation table and a blob descriptor based on history information, read the data in the memory based on the prefetch information, and read next data following the data in the memory based on the prefetch information for next operation of the neural network processor while the data is used by the neural network processor.

12 . The system of claim 11 , wherein the prefetch information includes first prefetch information about a source address of the data and a size of the data and second prefetch information about a source address of the next data and a size of the next data.

13 . The system of claim 12 , wherein the memory controller is further configured to access the source address of the data included in the first prefetch information to transfer the data to the neural network processor and access the source address of the next data included in the second prefetch information to transfer the next data to the neural network processor.

14 . The system of claim 12 , wherein the memory controller is further configured to store the updated data in the memory according to a target address corresponding to the source address of the data when the updated data is archived in a prefetch buffer.

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Kim Byung Jo
• Lee Joo Hyun
• Kim Seong Min
• Kim Ju-Yeob
• Kim Jin Kyu
• Lee Mi Young

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06F2212/6022

  Using a prefetch buffer or dedicated prefetch cache · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title