Resistive processing unit with hysteretic updates for neural network training

Patent metadata

Field	Value
Publication number	US-10755170-B2
Application number	US-201715446264-A
Country	US
Kind code	B2
Filing date	Mar 1, 2017
Priority date	Mar 1, 2017
Publication date	Aug 25, 2020
Grant date	Aug 25, 2020

Abstract

A technique relates a resistive processing unit (RPU) array. A set of conductive column wires are configured to form cross-points at intersections between the set of conductive row wires and a set of conductive column wires. Two-terminal RPUs are hysteretic such that the two-terminal RPUs each have a conductance state defined by hysteresis, where a two-terminal RPU of the two-terminal RPUs is located at each of the cross-points.

First claim

What is claimed is:

1. A resistive processing unit (RPU) array comprising:
a set of conductive row wires;
a set of conductive column wires configured to form a plurality of cross-points at intersections between the set of conductive row wires and the set of conductive column wires; and
a plurality of two-terminal RPUs that are hysteretic such that the plurality of two-terminal RPUs each have a conductance state defined by hysteresis, wherein a two-terminal RPU of the plurality of two-terminal RPUs is located at each of the plurality of cross-points, one terminal of the two-terminal RPU being coupled to a stochastic translator and another terminal of the two-terminal RPU being coupled to another stochastic translator.

2. The RPU array of claim 1 , wherein the hysteresis of the plurality of two-terminal RPUs is configured to cause a change in the conductance state to have a delay for a predefined amount of pulses.

3. The RPU array of claim 2 , wherein the predefined amount of pulses is a number of update pulses.

4. The RPU array of claim 3 , wherein the update pulses are configured to be applied to both the set of conductive row wires and the set of conductive column wires.

5. The RPU array of claim 2 , wherein the change in the conductance state is defined as a switch in the conductance state from a first direction to a second direction or a switch in the conductance state from the second direction to the first direction.

6. The RPU array of claim 5 , wherein the delay is equal for the switch in the conductance state from the first direction to the second direction and for the switch in the conductance state from the second direction to the first direction.

7. The RPU array of claim 3 , wherein the update pulses switch between a positive pulse and a negative pulse.

8. The RPU array of claim 7 , wherein the positive pulse causes the first direction of the conductance state and the negative pulse causes the second direction of the conductance state.

9. The RPU array of claim 1 , wherein:
the plurality of two-terminal RPUs are imbalanced devices having an imbalance in the conductance state between slopes of a first direction and a second direction; and
the hysteresis in the plurality of two-terminal RPUs decreases the imbalance.

10. A method of forming a resistive processing unit (RPU) array, the method comprising:
providing a set of conductive row wires;
providing a set of conductive column wires configured to form a plurality of cross-points at intersections between the set of conductive row wires and the set of conductive column wires; and
providing a plurality of two-terminal RPUs that are hysteretic such that the plurality of two-terminal RPUs each have a conductance state defined by hysteresis, wherein a two-terminal RPU of the plurality of two-terminal RPUs is located at each of the plurality of cross-points, one terminal of the two-terminal RPU being coupled to a stochastic translator and another terminal of the two-terminal RPU being coupled to another stochastic translator.

11. The method of claim 10 , wherein the hysteresis of the plurality of two-terminal RPUs is configured to cause a change in the conductance state to have a delay for a predefined amount of pulses.

12. The method of claim 11 , wherein the predefined amount of pulses is a number of update pulses.

13. The method of claim 12 , wherein the update pulses are configured to be applied to both the set of conductive row wires and the set of conductive column wires.

14. The method of claim 11 , wherein the change in the conductance state is defined as a switch in the conductance state from a first direction to a second direction or a switch in the conductance state from the second direction to the first direction.

15. The method of claim 14 , wherein the delay is equal for the switch in the conductance state from the first direction to the second direction and for the switch in the conductance state from the second direction to the first direction.

16. The method of claim 12 , wherein the update pulses switch between a positive pulse and a negative pulse.

17. The method of claim 16 , wherein the positive pulse causes the first direction of the conductance state and the negative pulse causes the second direction of the conductance state.

18. The method of claim 10 , wherein:
the plurality of two-terminal RPUs are imbalanced devices having an imbalance in the conductance state between slopes of a first direction and a second direction;
the hysteresis in the plurality of two-terminal RPUs decreases the imbalance.

19. A method for hysteresis operation, the method comprising:
causing update pulses to be received by at least one hysteretic resistive processing unit (RPU), one terminal of the at least one hysteretic RPU being coupled to a stochastic translator and another terminal of the at least one hysteretic RPU being coupled to another stochastic translator; and
causing the at least one hysteretic RPU to have a change in a conductance state in response to the update pulses, wherein the change in the conductance state has a delay for a predefined amount of the update pulses.

20. The method of claim 19 , wherein the change in the conductance state is a switch in the conductance state from a first direction to a second direction or a switch in the conductance state from the second direction to the first direction.

Assignees

• IBM

Inventors

• Gokmen Tayfun
• Tromp Rudolf M

Classifications

• G06N3/065 Primary

  Analogue means · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/0635 Primary

  Physics · mapped topic