Edge-side federated learning for anomaly detection

Patent metadata

Field	Value
Publication number	US-12340308-B2
Application number	US-202217695325-A
Country	US
Kind code	B2
Filing date	Mar 15, 2022
Priority date	Aug 6, 2020
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

Methods and systems for training a neural network include collecting model exemplar information from edge devices, each model exemplar having been trained using information local to the respective edge devices. The collected model exemplar information is aggregated together using federated averaging. Global model exemplars are trained using federated constrained clustering. The trained global exemplars are transmitted to respective edge devices.

First claim

What is claimed is:

1. A method for training a neural network, comprising:
training an edge model exemplar at an edge device, using an initialized global model exemplar, based on information collected at the edge device, including optimizing an objective function:

min

θ
,
C

⁢

-

1
n

⁢

∑

i
=
1

n

⁢

K
⁢

L
⁡

(

p
i

⁢




⁢

q
i

)

-

α
T

⁢

⁢
log
⁢

⁢

(

1
n

⁢

∑

i
=
1

n

⁢

q
i

)

+

1
⁢

/

⁢
n
⁢

∑

i
=
1

n

⁢

M
⁡

(

X
i

)

where n is a number of multivariate time series segments X i , θ is a set of parameters for the neural network to be learned, C is a set of edge model exemplars, KL(·) is a Kullback-Leibler divergence, p i is a target cluster membership vector for an i th locally gathered information, q i is a cluster membership vector for the i th locally gathered information, α is a prior distribution over the edge model exemplars, and M(X i ) is a term that preserves local similarity of an original feature space;
transmitting the edge model exemplar to a server without transmitting the information collected at the edge device;
receiving an updated global model exemplar that is based on the edge model exemplar and at least one other model exemplar from another edge device; and
retraining the edge model exemplar using the updated global model exemplar.

2. The method of claim 1 , wherein the updated global model exemplar is a federated average of the edge model exemplar and the at least one other model exemplar.

3. The method of claim 2 , wherein the federated average is an element-wise average of exemplars.

4. The method of claim 1 , further comprising repeating the transmitting, receiving, and retraining based on additional information collected at the edge device.

5. The method of claim 1 , wherein the edge model exemplar includes the neural network, which includes a bidirectional long-short term memory layer.

6. The method of claim 1 , further comprising determining an anomaly score using the retrained edge model exemplar based on the information gathered at the edge device.

7. The method of claim 6 , wherein determining the anomaly score is based on a similarity between new information and existing exemplars.

8. The method of claim 1 , wherein the retrained edge model exemplar recognizes operating conditions from cyber-physical systems associated with a plurality of edge devices.

9. A system for training a neural network, comprising: a hardware processor; and a memory that stores a computer program, which, when executed by the hardware processor, causes the hardware processor to:
train an edge model exemplar at an edge device, using an initialized global model exemplar, based on information collected at the edge device, including optimization of an objective function:

min

θ
,
C

⁢

-

1
n

⁢

∑

i
=
1

n

⁢

K
⁢

L
⁡

(

p
i

⁢




⁢

q
i

)

-

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Song Dongjin
• Chen Yuncong
• Lumezanu Cristian
• Mizoguchi Takehiko
• Chen Haifeng
• Zhu Wei

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title