Resistive processing unit architecture with separate weight update and inference circuitry

Patent metadata

Field	Value
Publication number	US-2019318239-A1
Application number	US-201815954170-A
Country	US
Kind code	A1
Filing date	Apr 16, 2018
Priority date	Apr 16, 2018
Publication date	Oct 17, 2019
Grant date	—

Abstract

Systems and methods are provided to perform weight update operations in a resistive processing unit (RPU) system to update weight values of RPU devices comprising tunable resistive device. A weight update operation for a given RPU device includes maintaining a weight update accumulation value for the RPU device, adjusting the weight update accumulation value by one unit update value in response to a detected coincidence of stochastic bits streams of input vectors applied on an update row and update column control lines connected to the RPU device, generating a weight update control signal in response to the accumulated weight value reaching a predefined threshold value, and adjusting a conductance level of the tunable resistive device by one unit conductance value in response to the weight update control signal, wherein the one unit conductance value corresponds to one unit weight value of the RPU device.

First claim

We claim:

1 . A resistive processing unit (RPU) device, comprising:
weight update accumulation circuitry having a first input connected to an update row control line, and a second input connected to an update column control line; weight update control circuitry connected to an output of the weight update accumulation circuitry; and a tunable resistive device coupled to the weight update control circuitry, and connected to a read row control line and a read column control line; wherein the weight update accumulation circuitry is configured to (i) maintain a weight update accumulation value h ij and adjust the weight update accumulation value h ij by one unit update value Δh in response to each detected coincidence of stochastic bits streams of input vectors applied on the update row and update column control lines, and (ii) output a weight update control signal to the weight update control circuitry in response to the accumulated weight value h ij reaching a predefined threshold value; and wherein the weight update control circuitry is configured to adjust a conductance level of the tunable resistive device by one unit conductance value Δg in response to the weight update control signal output from the weight update accumulation circuitry, wherein the one unit conductance value Δg corresponds to one unit weight value Δw of the RPU device.

2 . The RPU device of claim 1 , wherein the weight update accumulation circuitry is configured to perform accumulated weight update operations concurrently with a read operation in which a weight value w ij of the RPU device is sensed by applying read control voltages on the read row and read column control lines connected to the tunable resistive device.

3 . The RPU device of claim 1 , wherein the weight update accumulation circuitry and the weight update control circuitry are configured to receive a programming polarity control signal and adjust the weight update accumulation value and the conductance level of the tunable resistive device by one of (i) incrementing and (ii) decrementing the weight update accumulation value and the conductance level, depending on a polarity of the programming polarity control signal.

4 . The RPU device of claim 1 , wherein the weight update accumulation circuitry comprises:
a coincidence detector having inputs connected to the update row and update column control lines; and a bidirectional digital counter having a data input port connected to an output of the coincidence detector; wherein the coincidence detector is configured to output a detection signal to the data input port of the bidirectional digital counter in response to each detected coincidence of the stochastic bits streams of the input vectors applied on the update row and update column control lines; and wherein the bidirectional digital counter is configured to (i) adjust a count value in response the detection signal input to the data input port, and (ii) output the weight update control signal to the weight update control circuitry in response to the count value reaching a predefined count value.

5 . The RPU device of claim 4 , wherein the weight update control circuitry comprises pulse generator circuitry that is configured to generate a programming voltage pulse to adjust a conductance level of the tunable resistive device by one unit conductance value Δg.

6 . The RPU device of claim 5 , wherein the pulse generator circuitry is configured to generate a reset control signal that is input to a reset control port of the bidirectional digital counter to reset a count value of the bidirectional digital counter to a predefined initialization count value following adjustment of the conductance level of the tunable resistive device.

7 . The RPU device of claim 4 , wherein the predefined count value comprises one of a minimum count value and a maximum count value of the bidirectional digital counter.

8 . The RPU device of claim 4 , wherein the coincidence detector comprises an AND logic circuit.

9 . The RPU device of claim 1 , wherein the weight update control circuitry comprises pulse generator circuitry, and wherein the tunable resistive device comprises a tunable memristive device, wherein the pulse generator circuitry is configured to applying a programming voltage pulse across the tunable memristive device to adjust a conductance level of the tunable memristive device by one unit conductance value Δg.

10 . The RPU device of claim 9 , wherein the weight update control circuitry comprises a first switch and a second switch, wherein the first switch and the second switch are responsive to a path enable control signal to selectively connect the tunable memristive device across first and second output terminals of the pulse generator circuitry to perform a weight update process, and to selectively connect the tunable memristive device to the read row and read column control lines to perform a weight read operation.

11 . The RPU device of claim 9 , wherein the pulse generator circuitry comprises a local dedicated pulse generator circuit that is located within a footprint area of the RPU device.

12 . The RPU device of claim 9 , wherein the pulse generator circuitry comprises a global pulse generator circuit that is located outside a footprint area of the RPU device, wherein the global pulse generator circuit is shared with at least one other RPU device in an array of RPU devices.

13 . The RPU device of claim 1 , wherein the weight update control circuitry comprises a first current source, a second current source, and a capacitor, and wherein the tunable resistive device comprises a read transistor;
wherein the capacitor is configured to store a weight voltage for the RPU device; wherein the read transistor comprises a gate terminal, a first source/drain terminal, and a second source/drain terminal, wherein the gate terminal is connected to the capacitor, wherein the first source/drain terminal is connected to read row control line, and wherein the second source/drain terminal is connected to the read column control line; wherein the read transistor is configured to generate a weight current in response to the weight voltage of the capacitor applied to the gate terminal of the read transistor, in response to read control signals applied to the read row and read column control lines; wherein in response to the weight update control signal output from the weight update accumulation circuitry, the weight update control circuitry is configured to one of (i) selectively activate the first current source to apply a charging current to the capacitor to increase a weight voltage of the capacitor by one unit voltage change +ΔV and (ii) selectively activate the second current source to apply a discharging current to the capacitor to decrease a weight voltage of the capacitor by one unit voltage change −ΔV.

14 . An RPU crossbar array system comprising the RPU device of claim 1 .

15 . A method comprising:
performing a weight update operation to update a weight value of a resistive processing unit (RPU) device comprising a tunable resistive device, wherein the weight update operation comprises: maintaining a weight update accumulation value h ij for the RPU device; adjusting the weight update accumulation value h ij by one unit update value Δh in response to a detected coincidence of stochastic bits streams of input vectors applied on an update row and update column control lines connected to the RPU device; generating a weight update control signal in response to the accumulated weight value h ij reaching a predefined threshold value; and adjusting a conductance

Assignees

• IBM

Inventors

• Kim Seyoung
• Gokmen Tayfun

Classifications

• G06N3/065 Primary

  Analogue means · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0635

  Physics · mapped topic

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title