Distributed memory-augmented neural network architecture

Patent metadata

Field	Value
Publication number	US-2021311874-A1
Application number	US-202016838294-A
Country	US
Kind code	A1
Filing date	Apr 2, 2020
Priority date	Apr 2, 2020
Publication date	Oct 7, 2021
Grant date	—

Abstract

A method for using a distributed memory device in a memory augmented neural network system includes receiving, by a controller, an input query to access data stored in the distributed memory device, the distributed memory device comprising a plurality of memory banks. The method further includes determining, by the controller, a memory bank selector that identifies a memory bank from the distributed memory device for memory access, wherein the memory bank selector is determined based on a type of workload associated with the input query. The method further includes computing, by the controller and by using content based access, a memory address in the identified memory bank. The method further includes generating, by the controller, an output in response to the input query by accessing the memory address.

First claim

What is claimed is:

1 . A method for using a distributed memory device in a memory augmented neural network system, the method comprising:
receiving, by a controller, an input query to access data stored in the distributed memory device, the distributed memory device comprising a plurality of memory banks; determining, by the controller, a memory bank selector that identifies a memory bank from the distributed memory device for memory access, wherein the memory bank selector is determined based on a type of workload associated with the input query; computing, by the controller and by using content based access, a memory address in the identified memory bank; and generating, by the controller, an output in response to the input query by accessing the memory address.

2 . The method of claim 1 , wherein the memory banks in the distributed memory device are heterogeneous, with a first memory bank in the distributed memory device having attributes that are distinct from those of a second memory bank in the distributed memory device.

3 . The method of claim 1 , wherein the memory banks in the distributed memory device are homogenous, with the memory banks having substantially same attributes.

4 . The method of claim 1 , wherein the computing the memory address comprises: sending the memory bank selector as an input signal to a multiplexer that couples the identified memory bank with a weighting unit that computes the memory address.

5 . The method of claim 4 , wherein the weighting unit computes the memory address using a key strength parameter (( 3 ) and a key vector (k t ), both of which are computed by the controller based on the content based addressing.

6 . The method of claim 4 , wherein the weighting unit computes a weight vector as Softmax(K[k t ⊙s t , ⊙s t ]·β), wherein K is a similarity function based on the key vector k t , a memory matrix of the identified memory bank, a key strength parameter β, and the memory bank selector s t .

7 . The method of claim 1 , wherein each of the memory banks in the distributed memory device is represented by a distinct memory matrix .

8 . A neural network system, comprising:
a distributed memory device comprising a plurality of memory banks; and a controller coupled with the distributed memory device, the controller configured to perform a method to access data stored in the distributed memory device, the method comprising:
receiving an input query to access data stored in the distributed memory device;
determining a memory bank selector that identifies a memory bank from the distributed memory device for memory access, wherein the memory bank selector is determined based on a type of workload associated with the input query;
computing, by using content based access, a memory address in the identified memory bank; and
generating an output in response to the input query by accessing the memory address.

9 . The neural network system of claim 8 , wherein the memory banks in the distributed memory device are heterogeneous, with a first memory bank in the distributed memory device having attributes that are distinct from those of a second memory bank in the distributed memory device.

10 . The neural network system of claim 8 , wherein the memory banks in the distributed memory device are homogenous, with the memory banks having substantially same attributes.

11 . The neural network system of claim 8 , wherein the computing the memory address comprises: sending the memory bank selector as an input signal to a multiplexer that couples the identified memory bank with a weighting unit that computes the memory address.

12 . The neural network system of claim 11 , wherein the weighting unit computes the memory address using a key strength parameter (β) and a key vector (k t ), both of which are computed by the controller based on the content based addressing.

13 . The neural network system of claim 11 , wherein the weighting unit computes a weight vector as Softmax(K[k t ⊙s t , ⊙s t ]·β), wherein K is a similarity function based on the key vector k t , a memory matrix of the identified memory bank identified, a key strength parameter β, and the memory bank selector s t .

14 . The neural network system of claim 8 , wherein each of the memory banks in the distributed memory device is represented by a distinct memory matrix .

15 . A memory address determination apparatus, comprising:
a selection unit; and a weighting unit; wherein, the memory address determination apparatus is configured to perform a method to access data stored in a distributed memory device, the method comprising:
receiving, by the selection unit, a memory bank selector that identifies a memory bank from the distributed memory device for a memory access, the memory bank selector determined based on a type of workload associated with an input query received by a controller of a neural network; and
computing, by the weighting unit, a memory address, using content based access, in the memory bank identified, the memory address is used to access data from the distributed memory to generate an output in response to the input query.

16 . The memory address determination apparatus of claim 15 , wherein the memory banks in the distributed memory device are heterogeneous, with a first memory bank in the distributed memory device having attributes that are distinct from those of a second memory bank in the distributed memory device.

17 . The memory address determination apparatus of claim 15 , wherein the memory banks in the distributed memory device are homogenous, with the memory banks having substantially same attributes.

18 . The memory address determination apparatus of claim 15 , wherein the computing the memory address comprises: sending the memory bank selector as an input signal to a multiplexer that couples the identified memory bank with a weighting unit that computes the memory address.

19 . The memory address determination apparatus of claim 18 , wherein the weighting unit computes a weight vector as Softmax(K[k t ⊙s t , ⊙s t ]·β), wherein K is a similarity function based on the key vector k t , a memory matrix of the identified memory bank identified, a key strength parameter β, and the memory bank selector s t .

20 . The memory address determination apparatus of claim 15 , wherein each of the memory banks in the distributed memory device is represented by a distinct memory matrix .

21 . A computer-implemented method, comprising:
receiving, by a controller neural network, an input; generating, by the controller neural network, parameters including (i) a key vector and (ii) a selector vector to interface with a plurality of memory banks that are in communication with the controller neural network via a memory subsystem; selecting, by the memory subsystem, at least one of the memory banks for the controller neural network to access, using the selector vector and the key vector; and accessing, by the controller neural network, a memory address from the selected memory bank using the key vector.

22 . The computer-implemented method of claim 21 , wherein one or more of the memory banks, the subsystem, and the controller neural network comprise field programmable gate array (FPGA) devices.

23 . The computer-implemented method of claim 21 , wherein the accessing the memory address from the selected memory bank comprises:
generating a memory query vector by reading data from the selected memory ba

Assignees

• IBM

Inventors

• Ozcan Ahmet Serkan
• Kornuta Tomasz
• Radens Carl
• Antoine Nicolas

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/063

  using electronic means · CPC title