Privacy-preserving federated learning

Patent metadata

Field	Value
Publication number	US-12160504-B2
Application number	US-201916682927-A
Country	US
Kind code	B2
Filing date	Nov 13, 2019
Priority date	Nov 13, 2019
Publication date	Dec 3, 2024
Grant date	Dec 3, 2024

Abstract

A plurality of public encryption keys are distributed to a plurality of participants in a federated learning system, and a first plurality of responses is received from the plurality of participants, where each respective response of the first plurality of responses was generated based on training data local to a respective participant of the plurality of participants and is encrypted using a respective public encryption key of the plurality of public encryption keys. A first aggregation vector is generated based on the first plurality of responses, and a first private encryption key is retrieved using the first aggregation vector. An aggregated model is then generated based on the first private encryption key and the first plurality of responses.

First claim

What is claimed is:

1. A method, comprising:
facilitating distribution of a plurality of public encryption keys to a plurality of participants in a federated learning system, wherein:
the plurality of public encryption keys are generated based on a master public encryption key and a master private encryption key,
the master public encryption key and the master private encryption key are generated by a trusted entity by applying an encryption function using one or more defined key provisioning parameters indicating at least one of: (i) a number of the plurality of participants or (ii) a maximum number of expected participants as inputs to the encryption function, and
the key provisioning parameters are determined by one or more of the plurality of participants;

receiving a first plurality of responses from the plurality of participants, wherein each respective response of the first plurality of responses comprises machine learning model parameters determined based on training a local machine learning model using data local to a respective participant of the plurality of participants, and wherein each respective response is encrypted using a respective public encryption key of the plurality of public encryption keys;
generating a first aggregation vector based on the first plurality of responses;
upon determining the first aggregation vector satisfies one or more security criteria, retrieving a first private encryption key from the trusted entity using the first aggregation vector; and
generating an aggregated machine learning model based on the first private encryption key and the first plurality of responses.

2. The method of claim 1 , the method further comprising:
determining that a first participant of the plurality of participants has ceased participation in the federated learning system;
receiving a second plurality of responses from one or more of the plurality of participants, wherein each respective response of the second plurality of responses is encrypted using a respective public encryption key of the plurality of public encryption keys;
generating a second aggregation vector based on the second plurality of responses, wherein the second aggregation vector excludes the first participant;
retrieving a second private encryption key using the second aggregation vector; and
refining the aggregated model based on the second private encryption key and the second plurality of responses.

3. The method of claim 1 , the method further comprising:
determining that a new participant is beginning participation in the federated learning system, wherein the plurality of public encryption keys includes a first public encryption key that is not in use by any of the plurality of participants; and
facilitating distribution of the first public encryption key to the new participant.

4. The method of claim 1 , wherein the first aggregation vector defines a respective weighting for each of the plurality of participants.

5. The method of claim 1 , wherein the one or more security criteria comprises:
determining a number of non-zero entries in the first aggregation vector; and
confirming that the number of non-zero entries in the first aggregation vector exceeds a predefined threshold.

6. The method of claim 5 , wherein the one or more security criteria further comprises at least one of:
confirming that each non-zero entry in the first aggregation vector is equal to one divided by the number of non-zero entries in the first aggregation vector,
confirming that a weight assigned to each non-zero entry in the first aggregation vector exceeds a weight threshold, or
confirming that a difference between a first weight and a second weight does not exceed a difference threshold, wherein the first weight is assigned to a first non-zero entry, of the non-zero entries in the first aggregation vector, with a highest value, and the second weight is assigned to a second non-zero entry, of the non-zero entries in the first aggregation vector, with a lowest value.

7. The method of claim 1 , wherein the first private encryption key can be used to determine one or more aggregate values based on the first plurality of responses, wherein an individual value of each of the first plurality of responses remains encrypted.

8. One or more computer-readable storage media collectively containing computer program code that, when executed by operation of one or more computer processors, performs an operation comprising:
facilitating distribution of a plurality of public encryption keys to a plurality of participants in a federated learning system, wherein:
the plurality of public encryption keys are generated based on a master public encryption key and a master private encryption key, and
the master public encryption key and the master private encryption key are generated by a trusted entity by applying an encryption function using one or more defined key provisioning parameters indicating at least one of: (i) a number of the plurality of participants or (ii) a maximum number of expected participants as inputs to the encryption function, and
the key provisioning parameters are determined by one or more of the plurality of participants;

receiving a first plurality of responses from the plurality of participants, wherein each respective response of the first plurality of responses comprises machine learning model parameters determined based on training a local machine learning model using data local to a respective participant of the plurality of participants, and wherein each respective response is encrypted using a respective public encryption key of the plurality of public encryption keys;
generating a first aggregation vector based on the first plurality of responses;
upon determining the first aggregation vector satisfies one or more security criteria, retrieving a first private encryption key from the trusted entity using the first aggregation vector; and
generating an aggregated machine learning model based on the first private encryption key and the first plurality of responses.

9. The one or more computer-readable storage media of claim 8 , the operation further comprising:
determining that a first participant of the plurality of participants has ceased participation in the federated learning system;
receiving a second plurality of responses from one or more of the plurality of participants, wherein each respective response of the second plurality of responses is encrypted using a respective public encryption key of the plurality of public encryption keys;
generating a second aggregation vector based on the second plurality of responses, wherein the second aggregation vector excludes the first participant;
retrieving a second private encryption key using the second aggregation vector; and
refining the aggregated model based on the second private encryption key and the second plurality of responses.

10. The one or more computer-readable storage media of claim 8 , the operation further comprising:
determining that a new participant is beginning participation in the federated learning system, wherein the plurality of public encryption keys includes a first public encryption key that is not in use by any of the plurality of participants; and
facilitating distribution of the first public encryption key to the new participant.

11. The one or more computer-readable storage media of claim 8 ,
wherein the first aggregation vector defines a respective weighting for each of the plurality of participants.

12. The one or more computer-readable storage media of claim 8 , wherein the one or more security criteria comprises:
determining a number of non-zero entries in the first

Assignees

• IBM

Inventors

• Xu Runhua
• Baracaldo Angel Nathalie
• Zhou Yi
• Anwar Ali
• Ludwig Heiko H

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• H04L63/0442

  wherein the sending and receiving network entities apply asymmetric encryption, i.e. different keys for encryption and decryption (cryptographic mechanisms or cryptographic arrangements for public-key encryption H04L9/30) · CPC title

• H04L63/065

  for group communications (cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833) · CPC title

• H04L63/062

  for key distribution, e.g. centrally by trusted party (cryptographic mechanisms or cryptographic arrangements for key distribution involving a central third party H04L9/0819) · CPC title