Edge-side federated learning for anomaly detection

Patent metadata

Field	Value
Publication number	US-2022215256-A1
Application number	US-202217695325-A
Country	US
Kind code	A1
Filing date	Mar 15, 2022
Priority date	Aug 6, 2020
Publication date	Jul 7, 2022
Grant date	—

Abstract

Methods and systems for training a neural network include collecting model exemplar information from edge devices, each model exemplar having been trained using information local to the respective edge devices. The collected model exemplar information is aggregated together using federated averaging. Global model exemplars are trained using federated constrained clustering. The trained global exemplars are transmitted to respective edge devices.

First claim

What is claimed is:

1 . A method for training a neural network, comprising:
training an edge model exemplar using an initialized global model exemplar, based on information collected at an edge device; transmitting the edge model exemplar to a server; receiving an updated global model exemplar that is based on the edge model exemplar and at least one other model exemplar from another edge device; and retraining the edge model exemplar using the updated global model exemplar.

2 . The method of claim 1 , wherein the updated global model exemplar is a federated average of the edge model exemplar and the at least one other model exemplar.

3 . The method of claim 2 , wherein the federated average is an element-wise average of exemplars.

4 . The method of claim 1 , wherein the information collected at the edge device is not transmitted to the server.

5 . The method of claim 1 , further comprising repeating the transmitting, receiving, and retraining based on additional information collected at the edge device.

6 . The method of claim 1 , wherein the edge model exemplar is a neural network including a bidirectional long-short term memory layer.

7 . The method of claim 1 , wherein training the edge model exemplar includes optimizing the objective function:

min

θ
,
C

⁢

-

1
n

⁢

∑

i
=
1

n

⁢

K
⁢

L
⁡

(

p
i

⁢




⁢

q
i

)

-

α
T

⁢

⁢
log
⁢

⁢

(

1
n

⁢

∑

i
=
1

n

⁢

q
i

)

+

1
⁢

/

⁢
n
⁢

∑

i
=
1

n

⁢

M
⁡

(

X
i

)

where θ is a set of parameters for a neural network to be learned, C is a set of edge model exemplars, KL(·) is the Kullback-Leibler divergence, p i is a target cluster membership vector for an i th locally gathered information, q i is a cluster membership vector for an i th locally gathered information, a is a prior distribution over the exemplars, and M(X i ) is a term that preserves local similarity of an original feature space.

8 . The method of claim 1 , further comprising determining an anomaly score using the retrained edge model exemplar based on the information gathered at the edge device.

9 . The method of claim 8 , wherein determining the anomaly score is based on a similarity between new information and existing exemplars.

10 . The method of claim 1 , wherein the retrained edge model exemplar recognizes operating conditions from cyber-physical systems associated with a plurality of edge devices.

11 . A system for training a neural network, comprising:
a hardware processor; and a memory that stores a computer program, which, when executed by the hardware processor, causes the hardware processor to:
train an edge model exemplar using an initialized global model exemplar, based on information collected at an edge device;
transmit the edge model exemplar to a server;
receive an updated global model exemplar that is based on the edge model exemplar and at least one other model exemplar from another edge device; and
retrain the edge model exemplar using the updated global model exemplar.

12 . The system of claim 11 , wherein the updated global model exemplar is a federated average of the edge model exemplar and the at least one other model exemplar.

13 . The system of claim 12 , wherein the federated average is an element-wise average of exemplars.

14 . The system of claim 11 , wherein the information collected at the edge device is not transmitted to the server.

15 . The system of claim 11 , wherein the computer program further causes the hardware processor to repeat the transmission, receipt, and retraining based on additional information collected at the edge device.

16 . The system of claim 11 , wherein the edge model exemplar is a neural network including a bidirectional long-short term memory layer.

17 . The system of claim 11 , wherein the computer program causes the hardware processor to optimize the objective function:

min

θ
,
C

⁢

-

1
n

⁢

Assignees

• Nec Lab America Inc

Inventors

• Song Dongjin
• Chen Yuncong
• Lumezanu Cristian
• Mizoguchi Takehiko
• Chen Haifeng
• Zhu Wei

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title