Killing asymmetric resistive processing units for neural network training

Patent metadata

Field	Value
Publication number	US-9715656-B1
Application number	US-201615262582-A
Country	US
Kind code	B1
Filing date	Sep 12, 2016
Priority date	Sep 12, 2016
Publication date	Jul 25, 2017
Grant date	Jul 25, 2017

Abstract

Technical solutions are described for improving efficiency of training a resistive processing unit (RPU) array using a neural network training methodology. An example method includes reducing asymmetric RPUs from the RPU array by determining an asymmetric value of an RPU from the RPU array, and burning the RPU in response to the asymmetry value being above a predetermined threshold. The RPU can be burned by causing an electric voltage across the RPU to be above a predetermined limit. The method further includes initiating the training methodology for the RPU array after the asymmetric RPUs from the RPU array are reduced.

First claim

What is claimed is:

1. A resistive processing unit (RPU) array comprising:
a set of conductive row wires;
a set of conductive column wires configured to form a plurality of crosspoints at intersections between the set of conductive row wires and the set of conductive column wires; and
a plurality of two-terminal RPUs, wherein a two-terminal RPU is located at each of the plurality of crosspoints, wherein a conduction state of an RPU identifies a weight of a training methodology applied to said RPU; and
wherein, the RPU array unit is configured to receive one or more electric signals that burn a selected subset of RPUs from the plurality of the RPUs, the selected subset of RPUs being selected based on corresponding asymmetry values being above a predetermined asymmetry threshold, and wherein the asymmetry value of an RPU from the plurality of RPUs is measured by:
measuring a first change in the conduction state of the RPU caused by a positive pulse;
measuring a second change in the conduction state of the RPU caused by a negative pulse; and
determining the asymmetry value of the RPU as a difference between the first change and the second change.

2. The resistive processing unit (RPU) array of claim 1 , wherein the one or more electric signals cause a voltage above a predetermined limit at the selected subset of RPUs.

3. The resistive processing unit (RPU) array of claim 1 , wherein the RPU array receives the one or more electric signals that burn the selected subset of RPUs before initiating the training methodology.

4. The resistive processing unit (RPU) array of claim 1 , wherein the change in the conduction state comprises a non-linear change based on at least one first encoded signal applied to a first terminal and at least one second encoded signal applied to the second terminal of the RPU.

5. The resistive processing unit (RPU) array of claim 1 , wherein the one or more electric signals are directed to the selected subset of RPUs based on corresponding crosspoints between the set of conductive row wires and the set of conductive column wires.

6. The resistive processing unit (RPU) array of claim 1 , wherein each RPU comprises:
a first terminal;
a second terminal; and
an active region having the conduction state;
wherein the active region is configured to locally perform a data storage operation of the training methodology; and
wherein the active region is further configured to locally perform a data processing operation of the training methodology.

7. The resistive processing unit (RPU) array of claim 1 , wherein the training methodology comprises one from a group of training methodologies consisting of:
an online neural network training;
a matrix inversion; and
a matrix decomposition.

8. A neuron control system facilitating training a resistive processing unit (RPU) array, the neuron control system comprising:
the RPU array, which comprises:
a set of conductive row wires;
a set of conductive column wires configured to form a plurality of crosspoints at intersections between the set of conductive row wires and the set of conductive column wires; and
a two-terminal RPU at each of the plurality of crosspoints, the two-terminal RPU comprising:
a first terminal;
a second terminal; and
an active region having a conduction state; and
wherein the active region is configured to effect a non-linear change in the conduction state based on at least one first encoded signal applied to the first terminal and at least one second encoded signal applied to the second terminal; and

a processor configured to control electric voltage across each RPU from the RPU array, wherein the processor is configured to:
reduce asymmetric RPUs from the RPU array by:
determine an asymmetric value of an RPU from the RPU array; and
burn the RPU in response to the asymmetry value being above a predetermined threshold by causing the electric voltage across the RPU to be above a predetermined limit, wherein the processor burns the RPU by sending the first encoded signal above a first predetermined limit and sending the second encoded signal below a second predetermined limit.

9. The neuron control system of claim 8 , wherein the non-linear change comprises a rectifying non-linear change or a saturating non-linear change.

10. The neuron control system of claim 8 , wherein the non-linear change comprises an exponential non-linear change.

11. The neuron control system of claim 8 , wherein the RPU that is burned is a first RPU from the RPU array, and wherein the RPU array further comprises a second RPU that is not burned, the second RPU comprising an active region having a conduction state,
wherein the active region is further configured to locally perform a data storage operation of a training methodology based at least in part on the non-linear change in the conduction state; and
wherein the active region is further configured to locally perform a data processing operation of the training methodology based at least in part on the non-linear change in the conduction state.

12. The neuron control system of claim 8 , wherein the processor is further configured to initiate training methodology for the RPU array, wherein the processor reduces the asymmetric RPUs from the RPU array prior to initiating the training methodology.

13. The neuron control system of claim 12 , wherein the training methodology comprises at least one of:
an online neural network training;
a matrix inversion; and
a matrix decomposition.

14. A non-transitory computer program product for training a resistive processing unit (RPU) array, the computer program product comprising computer readable storage medium with computer executable instructions embedded therein, wherein the computer readable storage medium comprises instructions to:
reduce asymmetric RPUs from the RPU array by:
determining an asymmetric value of an RPU from the RPU array; and
burning the RPU in response to the asymmetry value being above a predetermined threshold by causing an electric voltage across the RPU to be above a predetermined limit by sending a first encoded signal above a first predetermined limit and sending a second encoded signal below a second predetermined limit to a first terminal and a second terminal of the RPU respectively; and
initiate a training methodology for the RPU array after the asymmetric RPUs from the RPU array are reduced.

15. The non-transitory computer program product of claim 14 , wherein the RPU array comprises:
a set of conductive row wires;
a set of conductive column wires configured to form a plurality of crosspoints at intersections between the set of conductive row wires and the set of conductive column wires; and
a two-terminal RPU at each of the plurality of crosspoints, wherein the two-terminal RPU comprises the first terminal, the second terminal, and an active region having a conduction state, wherein the conduction state identifies a weight of the training methodology applied to the RPU.

Assignees

• IBM

Inventors

• Gokmen Tayfun

Classifications

• G06N3/065 Primary

  Analogue means · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/063 Primary

  using electronic means · CPC title