Privacy-preserving machine learning in the three-server model

Patent metadata

Field	Value
Publication number	US-11222138-B2
Application number	US-201817057574-A
Country	US
Kind code	B2
Filing date	Jul 17, 2018
Priority date	May 29, 2018
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

Methods and systems according to embodiments of the invention provide for a framework for privacy-preserving machine learning which can be used to obtain solutions for training linear regression, logistic regression and neural network models. Embodiments of the invention are in a three-server model, wherein data owners secret-share their data among three servers who train and evaluate models on the joint data using three-party computation (3PC). Embodiments of the invention provide for efficient conversions between arithmetic, binary, and Yao 3PC, as well as techniques for fixed-point multiplication and truncation of shared decimal values. Embodiments also provide customized protocols for evaluating polynomial piecewise functions and a three-party oblivious transfer protocol.

First claim

What is claimed is:

1. A method of performing privacy-preserving machine learning in a three-server model, the method comprising:
storing, by three training computers, secret-shared private data from a plurality of data clients, wherein each secret-shared data item of the secret-shared private data is represented by three parts, and wherein the secret-shared private data includes a set of training samples, each having features and an output Y, wherein the three training computers comprise a first training computer, a second training computer, and a third training computer;
initializing values for a set of weights for a machine learning model, the weights being secret-shared among the three training computers, wherein the weights and the features are stored as integers;
truncating, by the three training computers, a result of a multiplication of a secret-shared feature and a secret-shared weight as part of training the machine learning model, wherein the result comprises a first share, a second share, and a third share, wherein truncating includes:
generating, by the second training computer and the third training computer, a random value, wherein the random value is determined to be a truncated third share;
truncating, by the second training computer, a sum of the second share and the third share resulting in an intermediate value, wherein the intermediate value is subtracted by the random value resulting in a truncated second share;
transmitting, by the second training computer, the truncated second share to the first training computer; and
transmitting, by the first training computer, a truncated first share to the third training computer, the truncated first share generated by truncating the first share; and

training, by performing additional multiplications and truncations for secret-shared features of the set of training samples and secret-shared weights, a machine learning model for predicting the outputs Y of the set of training samples.

2. The method of claim 1 , wherein the random value is generated using a pseudorandom function and a secret key stored by the second training computer and the third training computer.

3. The method of claim 1 further comprising:
multiplying, by the three training computers, matrix-vectors X and Y such that half of the multiplications are done locally, and wherein each server shares a final result Z i with N communications.

4. The method of claim 1 , wherein the training samples are selected as part of a batch of training samples that are selected randomly, wherein the batch of training samples have a batch size B proportional to available bandwidth in a time for one round trip.

5. The method of claim 1 , wherein the training samples are selected as part of a batch of training samples that are selected randomly, wherein the batch of training samples have a batch size B based on a size of the training samples.

6. The method of claim 1 , further comprising:
reconstructing the weights using weight parts at each of the three training computers.

7. The method of claim 1 , further comprising:
receiving a new sample having d features;
applying, by each of the three training computers, weight parts of the machine learning model to obtain output parts; and
reconstructing, by the three training computers, the output parts to obtain a predicted output for the new sample.

8. A method of performing privacy-preserving machine learning in a three-server model, the method comprising:
storing, by three training computers, secret-shared private data from a plurality of data clients, wherein each secret-shared data item of the secret-shared private data is represented by three parts, and wherein the secret-shared private data includes a set of training samples, each having features and an output Y, wherein the three training computers comprise a first training computer, a second training computer, and a third training computer;
initializing values for a set of weights for a machine learning model, the weights being secret-shared among the three training computers, wherein the weights and the features are stored as integers;
truncating, by the three training computers, a result of a multiplication of a secret-shared feature and a secret-shared weight as part of training the machine learning model, wherein the result comprises a first share, a second share, and a third share, wherein truncating includes:
performing preprocessing, by the three training computers, resulting in a random arithmetic share and a truncated random arithmetic share for each of the three training computers;
computing, by each of the three training computers, a respective result share minus the random arithmetic share resulting in intermediate shares of an intermediate value;
revealing, by the three training computers, the intermediate shares to the three training computers, resulting in the three training computers storing the intermediate value;
truncating, by the three training computers, the intermediate value, resulting in a truncated intermediate value; and
computing, by each of the three training computers, the respective truncated random arithmetic share plus the truncated intermediate value, resulting in a truncated data item, wherein the truncated data item is secret-shared among the three training computers; and

training, by performing additional multiplications and truncations for secret-shared features of the set of training samples and secret-shared weights, a machine learning model for predicting the outputs Y of the set of training samples.

9. The method of claim 8 , wherein performing preprocessing further comprises:
determining, by the three training computers, a random binary share for each of the three training computers;
generating, by the three training computers, shares of two of three shares of the random binary share; and
determining, by the three training computers, a third share of the three shares of the random binary share, based on the random binary share and the shares of two of the three shares of the random binary share.

10. The method of claim 8 , wherein performing preprocessing further comprises:
determining, by the three training computers, a random binary share for each of the three training computers, wherein the random binary share is truncated to be a truncated random binary share;
generating, by a first training computer and a second training computer of the three training computers, shares of a second share of the random binary share and shares of a truncated second share of the truncated random binary share;
generating, by the second training computer and a third training computer of the three training computers, shares of a third share of the random binary share and shares of a truncated third share of the truncated random binary share;
revealing, by the three training computers, the shares of the second share and the shares of the truncated second share to the first training computer and the second training computer, and revealing the shares of the third share and shares of the truncated third share to the second training computer and the third training computer; and
computing, by the three training computers, shares of a first share of the random binary share and shares of a truncated first share of the truncated random binary share based on the random binary share, the truncated random binary share, the shares of the second share, the shares of the third share, the shares of the truncated second share, and the shares of the truncated third share using a ripple carry subtraction circuit and thereafter revealing the shares of the first share and the shares of the truncated first share to the first training computer and the third training computer

Assignees

• Visa Int Service Ass

Inventors

• Mohassel Payman
• Rindal Peter

Classifications

• A63B2244/10

  Combat sports · CPC title

• A63B69/34

  {Tackling, blocking or grappling dummies, e.g.} boxing or {wrestling or American-} football dummies · CPC title

• A63B21/0435

  One or both ends being anchored to a rotating element · CPC title

• A63B21/023

  Wound springs · CPC title

• A63B21/0552

  Elastic ropes or bands · CPC title