Detection of model attacks in distributed ai

Patent metadata

Field	Value
Publication number	US-2022156368-A1
Application number	US-202016952726-A
Country	US
Kind code	A1
Filing date	Nov 19, 2020
Priority date	Nov 19, 2020
Publication date	May 19, 2022
Grant date	—

Abstract

A method for detecting an attack on a distributed artificial intelligence deployment comprising a plurality of worker devices. Each of the plurality of worker devices comprises a local machine learning model. Each local machine learning model comprises a plurality of layers. The method comprises calculating a first inference from first input data using a first machine learning model comprising layers of the plurality of layers of one or more of the local machine learning models and calculating additional inferences from the first input data using one or more additional machine learning models. Each of the additional machine learning models comprises at least one of the layers used in the first machine learning model and at least one layer from the pluralities of layers of the one or more local machine learning models that is not used by the first machine learning model. The method further comprises calculating differences between the first inference and each of the one or more additional inferences.

First claim

1 . A method for detecting an attack on a distributed artificial intelligence deployment comprising a plurality of worker devices, each of the plurality of worker devices comprising a local machine learning model, each local machine learning model comprising a plurality of layers, the method comprising;
calculating a first inference from first input data using a first machine learning model comprising layers of the plurality of layers of one or more of the local machine learning models; calculating an additional inference from the first input data using one or more additional machine learning models, each of the additional machine learning models comprising at least one of the layers used in the first machine learning model and at least one layer from the pluralities of layers of the one or more local machine learning models that is not used by the first machine learning model; and calculating differences between the first inference and each of the one or more additional inferences.

2 . A method according to claim 1 wherein the first machine learning model is a first worker device's local machine learning model.

3 . A method according to claim 2 wherein each additional machine learning model is a distributed machine learning model, each distributed machine learning model comprising at least one layer of the plurality of layers of the local machine learning model of the first worker device and at least one layer of the plurality of layers of a local machine learning model of another worker device.

4 . A method according to claim 3 wherein each distributed machine learning model comprises an input layer of the first worker device's local machine learning model.

5 . A method according to claim 3 wherein each distributed machine learning model comprises an output layer of the first worker device's local machine learning model.

6 . A method according to claim 3 wherein each of the one or more distributed machine learning models comprises intermediate layers and all of the intermediate layers of each distributed machine learning model are intermediate layers of local models of different worker devices other than the first worker device.

7 . A method according to claim 3 wherein the local and distributed machine learning models are artificial neural networks.

8 . A method according to claim 7 wherein each of the local and distributed learning models comprises the same number of layers and the same number of neurons in the same respective layers.

9 . A method according to claim 1 wherein a plurality of additional inferences are calculated using a plurality of local models each comprising a different combination of layers.

10 . A method according to claim 7 wherein the plurality of additional models comprise each possible combination of layers including the input and output layers of the first worker device's local model and intermediate layers of different other worker devices' local models.

11 . A method according to claim 9 further comprising comparing each of the plurality calculated differences between the local inference and the plurality of additional inferences to a threshold, and identifying any of the plurality of local models which comprise layers in all of the additional models with which additional inferences differing from the local inference of the first worker device by more than the threshold were calculated.

12 . A method according to claim 11 wherein the distributed artificial intelligence deployment is a federated learning deployment.

13 . A method according to claim 12 comprising selectively aggregating all of the plurality of local models which do not comprise layers in all of the additional models with which additional inferences differing from the local inference of the first worker device by more than the threshold were calculated.

14 . A method according to claim 13 wherein the plurality of local models which do not comprise layers in all of the additional models with which additional inferences differing from the local inference of the first worker device by more than the threshold were calculated are selectively aggregated to define a new machine learning model, and wherein the method comprises replacing the local machine learning models which do not comprise layers in all of the additional machine learning models with which additional inferences differing from the local inference of the first worker device by more than the threshold were calculated with the new machine learning model.

15 . One or more non-transitory storage media comprising computer instructions executable by one or more processors, the computer instructions when executed by the one or more processors causing the processors to perform a method for detecting an attack on a distributed artificial intelligence deployment comprising a plurality of worker devices, each of the plurality of worker devices comprising a local machine learning model, each local machine learning model comprising a plurality of layers, the method comprising;
calculating a first inference from first input data using a first machine learning model comprising layers of the plurality of layers of one or more of the local machine learning models; calculating an additional inference from the first input data using one or more additional machine learning models, each of the additional machine learning models comprising at least one of the layers used in the first machine learning model and at least one layer from the pluralities of layers of the one or more local machine learning models that is not used by the first machine learning model; and calculating differences between the first inference and each of the one or more additional inferences.

16 . A worker device comprising an antenna, a memory and a processor, the memory comprising computer instructions executable by the processor, the computer instructions when executed by the processor causing the processor to perform a method for detecting an attack on a distributed artificial intelligence deployment comprising the worker device and one or more additional worker devices, each of the plurality of worker devices comprising a local machine learning model, each local machine learning model comprising a plurality of layers, the method comprising;
calculating a first inference from first input data using a first machine learning model comprising layers of the plurality of layers of one or more of the local machine learning models; calculating an additional inference from the first input data using one or more additional machine learning models, each of the additional machine learning models comprising at least one of the layers used in the first machine learning model and at least one layer from the pluralities of layers of the one or more local machine learning models that is not used by the first machine learning model; and calculating differences between the first inference and each of the one or more additional inferences.

17 . A worker device according to claim 16 , the device further configured to calculate the additional inference by transmitting data for partially calculating said inference to another worker device of the plurality of worker devices alongside an indication of a layer within the other worker device that is to be used for said partial calculation of said additional inference;
wherein said data is input data or data generated by a layer of the local model of the worker device.

18 . A worker device according to claim 16 , the device further configured to receive layers of local models of other worker devices

Assignees

• Toshiba Kk

Inventors

• Spyridopoulos Theo
• Khan Aftab

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N3/063

  using electronic means · CPC title