Stuck-at fault mitigation method for ReRAM-based deep learning accelerators

Patent metadata

Field	Value
Publication number	US-11714727-B2
Application number	US-202217581327-A
Country	US
Kind code	B2
Filing date	Jan 21, 2022
Priority date	Jan 29, 2021
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

A stuck-at fault mitigation method for resistive random access memory (ReRAM)-based deep learning accelerators, includes: confirming a distorted output value (Y0) due to a stuck-at fault (SAF) by using a correction data set in a pre-trained deep learning network, by means of ReRAM-based deep learning accelerator hardware; updating an average (μ) and a standard deviation (σ) of a batch normalization (BN) layer by using the distorted output value (Y0), by means of the ReRAM-based deep learning accelerator hardware; folding the batch normalization (BN) layer in which the average (μ) and the standard deviation (σ) are updated into a convolution layer or a fully-connected layer, by means of the ReRAM-based deep learning accelerator hardware; and deriving a normal output value (Y1) by using the deep learning network in which the batch normalization (BN) layer is folded, by means of the ReRAM-based deep learning accelerator hardware.

First claim

What is claimed is:

1. A stuck-at fault mitigation method for resistive random access memory (ReRAM)-based deep learning accelerators comprising: a stuck-at fault confirming step of confirming a distorted output value (Y0) due to a stuck-at fault (SAF) by using a correction data set in a pre-trained deep learning network, by means of ReRAM-based deep learning accelerator hardware;
an updating step of updating an average (μ) and a standard deviation (σ) of a batch normalization (BN) layer by using the distorted output value (Y0), by means of the ReRAM-based deep learning accelerator hardware;
a folding step of folding the batch normalization (BN) layer in which the average (μ) and the standard deviation (σ) are updated into a convolution layer or a fully-connected layer, by means of the ReRAM-based deep learning accelerator hardware; and
a deriving step of deriving a normal output value (Y1) by using the deep learning network in which the batch normalization (BN) layer is folded, by means of the ReRAM-based deep learning accelerator hardware.

2. The stuck-at fault mitigation method for the ReRAM-based deep learning accelerators of claim 1 , wherein the deep learning network derives at least one of the distorted output value (Y0) and the normal output value (Y1) by using

y
=

γ
⁡
(

x
-
μ

σ
2

-
ε

)

+
β

,

based on affine transformation, wherein, y is an output value, x is an input value, μ and σ are the average and the standard deviation as forward parameters, β and γ are a bias value and an affine transformation weight as backward parameters, and ε is a constant.

3. The stuck-at fault mitigation method for the ReRAM-based deep learning accelerators of claim 2 , wherein in the updating step, the updating for parameters other than the average (μ) and the standard deviation (σ) of the batch normalization (BN) layer does not occur.

4. The stuck-at fault mitigation method for the ReRAM-based deep learning accelerators of claim 1 , wherein in the updating step, when there is no batch normalization (BN) layer in the deep learning network, the batch normalization (BN) layer is added to one side of the convolution layer or the fully-connected layer so that the stuck-at fault is mitigated, and
in the folding step, the batch normalization (BN) layer is folded into the convolution layer or the fully-connected layer so that the deriving step is simplified.

Assignees

• Unist Academy Industry Res Corporation
• Univ California
• Univ King Abdullah Sci & Tech

Inventors

• Lee Jong Eun
• Lee Su Gil
• Jung Gi Ju
• Fouda Mohammed
• Kurdahi Fadi
• Eltawil Ahmed M

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G11C29/70

  Masking faults in memories by using spares or by reconfiguring · CPC title

• G11C29/028

  with adaption or trimming of parameters · CPC title

• G11C11/54

  using elements simulating biological cells, e.g. neuron · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title