Methods and apparatus for federated training of a neural network using trusted edge devices

Patent metadata

Field	Value
Publication number	US-11526745-B2
Application number	US-201815892138-A
Country	US
Kind code	B2
Filing date	Feb 8, 2018
Priority date	Feb 8, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Methods, apparatus, systems and articles of manufacture for federated training of a neural network using trusted edge devices are disclosed. An example system includes an aggregator device to aggregate model updates provided by one or more edge devices. The one or more edge devices to implement respective neural networks, and provide the model updates to the aggregator device. At least one of the edge devices to implement the neural network within a trusted execution environment.

First claim

What is claimed is:

1. An edge device for federated training of a neural network, the edge device comprising:
a local data throttler to determine whether to allow a new local data item to be incorporated into a training process of a neural network at the edge device based on the local data throttler determining whether a number of training rounds elapsed since local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies a threshold number of elapsed training rounds, the threshold number of elapsed training rounds determined based on an instruction received at the edge device from an aggregator device, the neural network implemented within a trusted execution environment of the edge device;
a hash ledger to store hashes corresponding to local data items that are permitted to be used in training of the neural network;
a model receiver to apply model parameters provided to the neural network by the aggregator device;
a neural network trainer to train the neural network to create a model update using local data items that have a corresponding hash stored in the hash ledger; and
a model update provider to provide the model update to the aggregator device.

2. The edge device of claim 1 , wherein the local data throttler is further to determine whether the new local data item is trusted.

3. The edge device of claim 2 , wherein the local data throttler is to determine that the new local data item is trusted when the new local data item originates from trusted hardware.

4. The edge device of claim 1 , further including a local data accesser to validate hashes of the local data items against previously stored hashes of the respective local data items stored in the hash ledger.

5. The edge device of claim 4 , wherein the validating is to prevent use of the local data items that have been modified since their corresponding hash was stored in the hash ledger.

6. The edge device of claim 1 , wherein the local data throttler is to commit the hashes stored in the hash ledger in response to determining that the number of training rounds elapsed since the local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies the threshold number of elapsed training rounds, and the neural network trainer is to train the neural network using the local data items that have a corresponding committed hash stored in the hash ledger.

7. The edge device of claim 1 , wherein at least one of the local data throttler, the model receiver, the neural network trainer, and the model update provider are implemented within the trusted execution environment of the edge device.

8. At least one tangible machine readable storage medium comprising instructions which, when executed, cause at least one processor of an edge device to at least:
determine whether to allow a new local data item to be incorporated into a training process of a neural network implemented at the edge device based on the at least one processor determining whether a number of training rounds elapsed since local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies a threshold number of elapsed training rounds, the threshold number of elapsed training rounds determined based on an instruction received at the edge device from an aggregator device, the neural network implemented within a trusted execution environment;
store, in response to the determination that the new local data item is to be incorporated into the training process of the neural network, a hash of the new local data item in a hash ledger;
apply model parameters to the neural network, the model parameters received from the aggregator device;
train the neural network to create a model update using local data items, the local data items having hashes stored in the hash ledger; and
provide the model update to the aggregator device.

9. The at least one machine-readable storage medium of claim 8 , wherein the instructions, when executed, cause the at least one processor to:
commit the hash stored in the hash ledger in response to the determination that the number of training rounds elapsed since the local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies the threshold number of elapsed training rounds; and
train the neural network using local data items having committed hashes stored in the hash ledger.

10. The at least one machine-readable storage medium of claim 9 , wherein the committing of the hash stored in the hash ledger is responsive to an instruction provided to the edge device by the aggregator device.

11. The at least one machine-readable storage medium of claim 8 , wherein the instructions, when executed, cause the at least one processor to validate hashes of the local data items against previously stored hashes of the respective local data items stored in the hash ledger.

12. The at least one machine-readable storage medium of claim 11 , wherein the validating is to prevent use of the local data items that have been modified since their corresponding hash was stored in the hash ledger.

13. The at least one machine-readable storage medium of claim 8 , wherein the instructions, when executed, cause the at least one processor to determine whether the new training data originates from trusted input hardware.

14. A method for federated training of a neural network, the method comprising:
determining, by executing an instruction with a processor of an edge device, whether to allow a new local data item to be incorporated into a training process of a neural network implemented at the edge device based on the processor of the edge device determining whether a number of training rounds elapsed since local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies a threshold number of elapsed training rounds, the threshold number of elapsed training rounds determined based on an instruction received at the edge device from an aggregator device, the neural network implemented within a trusted execution environment;
storing, in response to determining that the new local data item is to be incorporated into the training process of the neural network, a hash of the new local data item in a hash ledger;
applying model parameters to the neural network, the model parameters received from the aggregator device;
training the neural network to create a model update using local data items, the local data items having hashes stored in the hash ledger; and
providing the model update to the aggregator device.

15. The method of claim 14 , further including committing the hash stored in the hash ledger in response to determining that the number of training rounds elapsed since the local data was allowed to be incorporated into the training process of the neural network at the edge device satisfies the threshold number of elapsed training rounds, wherein the training of the neural network is performed using the local data items having committed hashes stored in the hash ledger.

16. The method of claim 15 , wherein the committing of the hash stored in the hash ledger is responsive to an instruction provided to the edge device by the aggregator device.

17. The method of claim 14 , further including validating hashes of the local data items against previously stored hashes of the respective local data items stored in the hash ledger.

18. The method of claim 17 , wherein the validating prevents use

Assignees

• Intel Corp

Inventors

• Sheller Micah
• Cornelius Cory
• Martin Jason
• Huang Yonghong
• Wang Shih-Han

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06F2221/034

  Test or assess a computer or a system · CPC title

• G06F21/554

  involving event detection and direct action · CPC title

• G06N3/063

  using electronic means · CPC title

• G06N3/098 Primary

  Distributed learning, e.g. federated learning · CPC title