Privacy enhancing deep learning cloud service using a trusted execution environment

Patent metadata

Field	Value
Publication number	US-11443182-B2
Application number	US-201816016752-A
Country	US
Kind code	B2
Filing date	Jun 25, 2018
Priority date	Jun 25, 2018
Publication date	Sep 13, 2022
Grant date	Sep 13, 2022

Abstract

Mechanisms are provided to implement an enhanced privacy deep learning system framework (hereafter “framework”). The framework receives, from a client computing device, an encrypted first subnet model of a neural network, where the first subnet model is one partition of multiple partitions of the neural network. The framework loads the encrypted first subnet model into a trusted execution environment (TEE) of the framework, decrypts the first subnet model, within the TEE, and executes the first subnet model within the TEE. The framework receives encrypted input data from the client computing device, loads the encrypted input data into the TEE, decrypts the input data, and processes the input data in the TEE using the first subnet model executing within the TEE.

First claim

What is claimed is:

1. A method, in a data processing system comprising at least one processor and at least one memory, the at least one memory comprising instructions that are executed by the at least one processor to configure the at least one processor to implement an enhanced privacy deep learning system framework, the method comprising:
receiving, by the enhanced privacy deep learning system framework, from a client computing device, an encrypted first subnet model of a neural network, wherein the first subnet model is one partition of multiple partitions of the neural network;
loading, by the enhanced privacy deep learning system framework, the encrypted first subnet model into a trusted execution environment of the enhanced privacy deep learning system framework;
decrypting, by the enhanced privacy deep learning system framework, the first subnet model within the trusted execution environment and executing the first subnet model within the trusted execution environment;
receiving, by the enhanced privacy deep learning system framework, encrypted input data from the client computing device;
loading, by the enhanced privacy deep learning system framework, the encrypted input data into the trusted execution environment; and
decrypting and processing, by the enhanced privacy deep learning system framework, the input data in the trusted execution environment using the first subnet model executing within the trusted execution environment, wherein the first subnet model is a FrontNet subnet model comprising an input layer of the neural network and one or more intermediate layers of the neural network model, and wherein the neural network comprises a second subnet model that is a BackNet subnet model comprising an output layer of the neural network that outputs result data, and one or more intermediate layers of the neural network model; and
outputting the result data to a deep learning system to perform a classification operation to classify the encrypted input data into one of a plurality of predefined classes.

2. The method of claim 1 , wherein a partition point in the neural network indicating a last intermediate layer to be included in the FrontNet subnet model is selected as an intermediate layer whose intermediate representation output does not contain sensitive information corresponding to an input to the neural network, and wherein subsequent intermediate layers and the output layer of the neural network are included in the BackNet subnet model.

3. The method of claim 1 , wherein the neural network is partitioned automatically using an automated partitioning tool that identifies an optimal partition point in the neural network at which to partition the neural network, wherein the optimal partition point identifies an intermediate layer at which to partition the neural network into the first subnet model and the second subnet model.

4. The method of claim 1 , wherein the processing of the input data in the trusted execution environment using the first subnet model executing within the trusted execution environment generates one or more intermediate representations of processing of the input data, and wherein the method further comprises:
inputting the one or more intermediate representations into the second subnet model of the neural network;
processing, by the second subnet model, the one or more intermediate representations to generate result data; and
outputting the result data.

5. The method of claim 1 , wherein the second subnet model executes outside the trusted execution environment.

6. The method of claim 4 , wherein the result data is a N-dimensional real-value vector that represents a probability distribution over N different possible classes, and wherein the method further comprises selecting a top-k classes with corresponding probability values from the N-dimensional real-value vector, to return to the client computing device.

7. The method of claim 1 , wherein the input data is an input image and the classification operation classifies the input image into one of a plurality of predefined classes.

8. The method of claim 1 , wherein the trusted execution environment prevents access to the decrypted first subnet model and decrypted input data from outside the trusted execution environment.

9. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement an enhanced privacy deep learning system framework that is configured to:
receive from a client computing device, an encrypted first subnet model of a neural network, wherein the first subnet model is one partition of multiple partitions of the neural network;
load the encrypted first subnet model into a trusted execution environment of the enhanced privacy deep learning system framework;
decrypt the first subnet model within the trusted execution environment and execute the first subnet model within the trusted execution environment;
receive encrypted input data from the client computing device;
load the encrypted input data into the trusted execution environment; and
decrypt and process the input data in the trusted execution environment using the first subnet model executing within the trusted execution environment, wherein the first subnet model is a FrontNet subnet model comprising an input layer of the neural network and one or more intermediate layers of the neural network model, and wherein the neural network comprises a second subnet model that is a BackNet subnet model comprising an output layer of the neural network that outputs result data, and one or more intermediate layers of the neural network model; and
output the result data to a deep learning system to perform a classification operation to classify the encrypted input data into one of a plurality of predefined classes.

10. The computer program product of claim 9 , wherein a partition point in the neural network indicating a last intermediate layer to be included in the FrontNet subnet model is selected as an intermediate layer whose intermediate representation output does not contain sensitive information corresponding to an input to the neural network, and wherein subsequent intermediate layers and the output layer of the neural network are included in the BackNet subnet model.

11. The computer program product of claim 9 , wherein the neural network is partitioned automatically using an automated partitioning tool that identifies an optimal partition point in the neural network at which to partition the neural network, wherein the optimal partition point identifies an intermediate layer at which to partition the neural network into the first subnet model and the second subnet model.

12. The computer program product of claim 9 , wherein the processing of the input data in the trusted execution environment using the first subnet model executing within the trusted execution environment generates one or more intermediate representations of processing of the input data, and wherein the method further comprises:
inputting the one or more intermediate representations into the second subnet model of the neural network;
processing, by the second subnet model, the one or more intermediate representations to generate result data; and
outputting the result data.

13. The computer program product of claim 9 , wherein the second subnet model executes outside the trusted execution environment.

14. The computer program product of claim 12 , wherein the result data is a N-dimensional real-value vector that represents a

Assignees

• IBM

Inventors

• Gu Zhongshu
• Huang Heqing
• Zhang Jialong
• Su Dong
• Pendarakis Dimitrios
• Molloy Ian M

Classifications

• G06N3/063 Primary

  using electronic means · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F21/602

  Providing cryptographic facilities or services · CPC title