Neural flow attestation

Patent metadata

Field	Value
Publication number	US-11783201-B2
Application number	US-202016750328-A
Country	US
Kind code	B2
Filing date	Jan 23, 2020
Priority date	Jan 23, 2020
Publication date	Oct 10, 2023
Grant date	Oct 10, 2023

Abstract

Mechanisms are provided to implement a neural flow attestation engine and perform computer model execution integrity verification based on neural flows. Input data is input to a trained computer model that includes a plurality of layers of neurons. The neural flow attestation engine records, for a set of input data instances in the input data, an output class generated by the trained computer model and a neural flow through the plurality of layers of neurons to thereby generate recorded neural flows. The trained computer model is deployed to a computing platform, and the neural flow attestation engine verifies the execution integrity of the deployed trained computer model based on a runtime neural flow of the deployed trained computer model and the recorded neural flows.

First claim

What is claimed is:

1. A method, in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions that are executed by the at least one processor to configure the at least one processor to implement a neural flow attestation engine, the method comprising:
inputting, by the neural flow attestation engine, input data to a trained computer model, wherein the trained computer model comprises a plurality of layers of neurons;
recording, by the neural flow attestation engine, for a set of input data instances in the input data, an output class generated by the trained computer model and a neural flow through the plurality of layers of neurons to thereby generate recorded neural flows, wherein the output class is one of a plurality of possible output classes;
deploying the trained computer model to a computing platform; and
verifying, by the neural flow attestation engine, an integrity of the deployed trained computer model based on a runtime neural flow of the deployed trained computer model and the recorded neural flows, wherein the verifying comprises:
determining deviations between the runtime neural flow and one or more recorded neural flows corresponding to a same output class as generated by the deployed trained computing model for the runtime input data; and
determining that the execution integrity of the deployed trained computer model has been compromised in response to the deviations satisfying a predetermined criterion.

2. The method of claim 1 , further comprising:
generating, by the neural flow attestation engine, for each class in the plurality of possible output classes, a neural flow model based on the recorded neural flows; and
storing, by the neural flow attestation engine, each neural flow model in a measurement database, wherein verifying the execution integrity of the deployed trained computer model comprises comparing the runtime neural flow to one or more of the stored neural flow models in the measurement database.

3. The method of claim 2 , wherein generating, for each class in the plurality of possible output classes, a neural flow model based on the recorded neural flows comprises, for each class, aggregating the recorded neural flows associated with that class by at least one of identifying a set of activated neurons and filters or training a machine learning model based on activated neurons and filters taking into account activations and frequency of activation occurrence.

4. The method of claim 2 , wherein the measurement database is stored on a tenant computing device associated with a provider of the trained computer model, and wherein the data processing system is a separate computing system of a cloud computing platform.

5. The method of claim 4 , wherein verifying the execution integrity of the deployed trained computer model comprises:
recording the runtime neural flow of the deployed trained computer model based on runtime input data processed by the deployed trained computer model;
transmitting, from the data processing system, the recorded runtime neural flow to the tenant computing device; and
comparing the recorded runtime neural flow with one or more stored neural flow models in the measurement database corresponding to a same class as a runtime output class generated by the deployed trained computer model for the runtime input data.

6. The method of claim 1 , further comprising:
in response to determining that the integrity of the deployed trained computer model has been compromised, retracting the deployed trained computer model from further access by users of the deployed trained computer model.

7. The method of claim 1 , wherein recording, for the set of input data instances in the input data, the output class generated by the trained computer model and the neural flow through the plurality of layers of neurons comprises:
generating a trusted execution environment (TEE) in the data processing system;
recording, by a neural flow recording component executing within the trusted execution environment, the neural flow through the plurality of layers of neurons for the input data instance; and
generating, by an attestation generation engine executing within the trusted execution environment, an attestation comprising the recorded neural flow through the plurality of layers of neurons for the input data instance and an output class generated by the deployed trained computer model for the input data instance.

8. The method of claim 7 , wherein recording, for the set of input data instances in the input data, the output class generated by the trained computer model and the neural flow through the plurality of layers of neurons further comprises:
generating, by the attestation generation engine, the attestation further based on a challenge provided by a computing device of a provider of the computer model;
encrypting, by the attestation generation engine, the attestation based on a security key exchanged between the computing device of the provider of the computer model and the TEE, to thereby generate an encrypted attestation; and
transmitting, by the data processing system, the encrypted attestation to the computing device of the provider of the computer model.

9. The method of claim 1 , wherein the computing platform is a cloud computing platform having one or more tenants.

10. A computer program product comprising a computer non-transitory readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a neural flow attestation engine, that performs the operations of:
inputting input data to a trained computer model, wherein the trained computer model comprises a plurality of layers of neurons;
recording, for a set of input data instances in the input data, an output class generated by the trained computer model and a neural flow through the plurality of layers of neurons to thereby generate recorded neural flows, wherein the output class is one of a plurality of possible output classes;
deploying the trained computer model to a computing platform; and
verifying an integrity of the deployed trained computer model based on a runtime neural flow of the deployed trained computer model and the recorded neural flows, wherein the verifying comprises:
determining deviations between the runtime neural flow and one or more recorded neural flows corresponding to a same output class as generated by the deployed trained computing model for the runtime input data; and
determining that the execution integrity of the deployed trained computer model has been compromised in response to the deviations satisfying a predetermined criterion.

11. The computer program product of claim 10 , wherein the neural flow attestation engine further performs the operations of:
generating, for each class in the plurality of possible output classes, a neural flow model based on the recorded neural flows; and
storing, each neural flow model in a measurement database, wherein verifying the integrity of the deployed trained computer model comprises comparing the runtime neural flow to one or more of the stored neural flow models in the measurement database.

12. The computer program product of claim 11 , wherein generating, for each class in the plurality of possible output classes, a neural flow model based on the recorded neural flows comprises, for each class, aggregating the recorded neural flows associated with that class by at least one of identifying a set of activated neurons and filters or training a machine learning model based on activ

Assignees

• IBM

Inventors

• Gu Zhongshu
• Shu Xiaokui
• Jamjoom Hani
• Ma Tengfei

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/10 Primary

  Interfaces, programming languages or software development kits, e.g. for simulating neural networks · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title