Method, apparatus and system for secure vertical federated learning

Patent metadata

Field	Value
Publication number	US-11836583-B2
Application number	US-202017014204-A
Country	US
Kind code	B2
Filing date	Sep 8, 2020
Priority date	Sep 9, 2019
Publication date	Dec 5, 2023
Grant date	Dec 5, 2023

Abstract

A machine learning model is learned using secure vertical federated learning by receiving, by a network machine learning model, from a plurality of private machine learning models, a set of private machine learning model outputs. The set of private machine learning model outputs is based on data owned exclusively by each of the plurality of private machine learning models. The set of private machine learning model machine learning outputs is aligned based on sample IDs of the data. The network machine learning model, a prediction, the prediction being the output of the network model based on the set of private machine learning model outputs. Transmitting, by the network model, the prediction, to one of the plurality of private machine learning models, the one of the plurality of private machine learning models comprising labels. Receiving, by the network model, from the one of the plurality of private machine learning models, a loss based on the labels and the prediction, and calculating a gradient based on the loss, and updating a parameter of the network model based on the loss.

First claim

We claim:

1. A method for operating a machine learning model, the method comprising:
receiving, by a server operating a central machine learning model g 0 , from a plurality of computing systems each operating a different respective one of a plurality of private machine learning models a set of outputs g 1 , . . . , g k ; a set of outputs o 1 , . . . , o k of the private machine learning models, wherein each one of the plurality of computing systems transmits a different respective member o j of the set of outputs for said receipt by the server, each respective member o j of the set of outputs being generated by a different respective one of the private machine learning models g j based on a different respective portion x i,j of an identified data sample x i , such that at least one respective portion x i,j is strictly contained within the data sample x i and each respective portion x i,j is contained within or is equal to the data sample x i , wherein the data sample x i is data for a single entity, the data sample consists of multiple features corresponding to different aspects of the single entity, and the different respective portions x i,j are different subsets of said multiple features, the set of outputs being aligned based on a sample ID identifying said data sample x i ;
providing, by the server, a prediction o 0 computed by the central machine learning model g 0 based on the set of outputs o 1 , . . . , o k in combination;
receiving, at the server, a loss, the loss computed based on a comparison of the prediction with a label corresponding to the data sample; and
initiating updating of current model parameters of the machine learning model based on the loss.

2. The method of claim 1 , further comprising computing, by the server, a gradient of the loss relative to model parameters of the central machine learning model, wherein said updating the current model parameters of the machine learning model being trained is based directly or indirectly on said gradient of the loss.

3. The method of claim 1 , wherein providing the prediction comprises transmitting the prediction to a designated computing system of the plurality of computing systems, said designated computing system storing the label, the loss is computed by the designated computing system, and wherein receiving the loss comprises receiving the loss from the designated computing system.

4. The method of claim 1 , wherein providing the prediction comprises providing the prediction to a training function of the server, said training function storing the label, and wherein receiving the loss comprises receiving the loss from the training function, the method further comprising: prior to said receiving the prediction, receiving, by the server from a designated computing system of the computing systems, the label and storing the label in a memory accessible to the server; and computing the loss using the training function.

5. The method of claim 1 , wherein the machine learning model being trained is the central machine learning model, the method further comprising, by the server, updating said current model parameters of the central machine learning model based on the loss.

6. The method of claim 1 , wherein the machine learning model being trained is one of the plurality of private machine learning models, wherein initiating updating of current model parameters of the machine learning model being trained comprises transmitting, by the server, an indication of the loss or an indication of a gradient of the loss to a computing system operating one of the plurality of private machine learning models.

7. The method of claim 6 , wherein said gradient of the loss is a gradient relative to model parameters of the central machine learning model, wherein said updating the current model parameters of said one of the private machine learning models is based directly or a gradient of the loss relative to the current model parameters, said gradient of the loss relative to the current model parameters computed from said gradient relative to model parameters of the central machine learning model using a computation implementing a chain rule of derivatives.

8. The method of claim 1 , further comprising repeating said method until a convergence condition is reached.

9. A server comprising a computer processor operatively coupled to memory, and a communication interface, the server operating a central machine learning model and configured to:
receive, from a plurality of computing systems each operating a different respective one of a plurality of private machine learning models g 1 , . . . , g k : a set of outputs o 1 , . . . , o k of the private machine learning models, wherein each one of the plurality of computing systems transmits a different respective member o j of the set of outputs for said receipt by the server, each respective member o j of the set of outputs being generated by a different respective one of the private machine learning models g j based on a different respective portion x i,j of an identified data sample x i , such that at least one respective portion x i,j is strictly contained within the data sample x i , and each respective portion x i,j is contained within or is equal to the data sample x i , wherein the data sample x i is data for a single entity, the data sample consists of multiple features corresponding to different aspects of the single entity, and the different respective portions x i,j are different subsets of said multiple features, the set of outputs being aligned based on a sample ID identifying said data sample x i ;
provide a prediction o 0 computed by the central machine learning model g 0 based on the set of outputs o 1 , . . . , o k in combination;
receive a loss computed based on a comparison of the prediction with a label corresponding to the data sample; and
initiate updating of current model parameters of the machine learning model based on the loss.

10. The server of claim 9 , further configured to compute a gradient of the loss relative to model parameters of the central machine learning model, wherein said updating the current model parameters of the machine learning model being trained is based directly or indirectly on said gradient of the loss.

11. The server of claim 9 , wherein providing the prediction comprises transmitting the prediction to a designated computing system of the plurality of computing systems, said designated computing system holding the label, the loss is computed by the designated computing system, and wherein receiving the loss comprises receiving the loss from the designated computing system.

12. The server of claim 9 , wherein providing the prediction comprises providing the prediction to a training function of the server, said training function holding the label, and wherein receiving the loss comprises receiving the loss from the training function, the server further configured to: prior to said receiving the prediction, receive, from a designated computing system of the computing systems, the label; store the label in a memory accessible to the server; and compute the loss using the training function.

13. The server of claim 9 , wherein the machine learning model being trained is the central machine learning model, the server further configured to update said current model parameters of the central machine learning model based on the loss.

14. The server of claim 9 , wherein the machine learning model being trained is one of the plurality of private machine learning models, wherein initiating updating of current model parameters of the machine learning model being trained comprises transmitting, by t

Assignees

• Huawei Cloud Computing Tech Co Ltd

Inventors

• Chu Lingyang
• Huang Yutao
• Zhang Yong
• Wang Lanjun

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06Q30/0201

  Market modelling; Market analysis; Collecting market data · CPC title

• G06N3/045 Primary

  Combinations of networks · CPC title