Parameter sharing in federated learning

Patent metadata

Field	Value
Publication number	US-11645582-B2
Application number	US-202016832809-A
Country	US
Kind code	B2
Filing date	Mar 27, 2020
Priority date	Mar 27, 2020
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

One embodiment provides a method for federated learning across a plurality of data parties, comprising assigning each data party with a corresponding namespace in an object store, assigning a shared namespace in the object store, and triggering a round of federated learning by issuing a customized learning request to at least one data party. Each customized learning request issued to a data party triggers the data party to locally train a model based on training data owned by the data party and model parameters stored in the shared namespace, and upload a local model resulting from the local training to a corresponding namespace in the object store the data party is assigned with. The method further comprises retrieving, from the object store, local models uploaded to the object store during the round of federated learning, and aggregating the local models to obtain a shared model.

First claim

What is claimed is:

1. A method for federated learning across a plurality of data parties, comprising:
assigning each data party of the plurality of data parties with a corresponding namespace in an object store, wherein the corresponding namespace in the object store that the data party is assigned with is unique to the data party;
assigning a shared namespace in the object store, wherein the shared namespace is shared by the plurality of data parties;
storing one or more model parameters in the shared namespace in the object store;
triggering a round of federated learning by issuing one or more customized learning requests to one or more of the plurality of data parties, wherein each customized learning request issued to each data party of the one or more data parties comprises a link to the object store, a file name to use that is indicative of the round of federated learning, and information identifying both the corresponding namespace in the object store that the data party is assigned with and the shared namespace in the object store, and the customized learning request triggers the data party to fetch the one or more model parameters from the shared namespace in the object store, locally train a model based on training data owned by the data party and the one or more model parameters, and upload using the file name a local model resulting from the local training to the corresponding namespace in the object store that the data party is assigned with;
retrieving, from the object store, at least one local model uploaded to the object store by at least one data party of the plurality of data parties during the round of federated learning; and
aggregating the at least one local model retrieved from the object store to obtain a shared model.

2. The method of claim 1 , wherein the shared model is uploaded to the shared namespace in the object store.

3. The method of claim 2 , wherein each data party of the plurality of data parties is notified of the shared model uploaded to the shared namespace in the object store.

4. The method of claim 2 , further comprising:
triggering a subsequent round of federated learning by issuing one or more subsequent customized learning requests to one or more of the plurality of data parties, wherein each subsequent customized learning request issued to each data party of the one or more data parties triggers the data party to retrieve the shared model from the shared namespace in the object store, locally train the shared model based on training data owned by the data party, and upload using a different file name that is indicative of the subsequent round of federated learning an updated version of a local model resulting from the local training to a corresponding namespace in the object store that the data party is assigned with.

5. The method of claim 1 , wherein triggering a round of federated learning comprises:
selecting a set of data parties; and
triggering the set of data parties to locally train a set of models in parallel by issuing a customized learning request to each data party of the set of data parties.

6. The method of claim 1 , wherein triggering a round of federated learning comprises:
triggering the plurality of data parties to locally train a plurality of models in parallel by issuing a customized learning request to each data party of the plurality of data parties.

7. The method of claim 1 , further comprising:
storing, for each data party of the one or more data parties, using multiple file names that are indicative of multiple rounds of federated learning, multiple versions of a local model uploaded to a corresponding namespace in the object store assigned to the data party during the multiple rounds of federated learning.

8. The method of claim 1 , further comprising:
storing multiple versions of the shared model uploaded to the shared namespace in the object store during multiple rounds of federated learning.

9. The method of claim 1 , further comprising:
receiving, from the object store, at least one notification of the at least one local model uploaded to the object store by the at least one data party during the round of federated learning.

10. The method of claim 1 , further comprising:
receiving, from the at least one data party, at least one notification of the at least one local model uploaded to the object store by the at least one data party during the round of federated learning.

11. The method of claim 1 , further comprising:
periodically checking the object store for one or more local models uploaded to the object store by one or more data parties in accordance with a schedule.

12. A system for federated learning across a plurality of data parties, comprising:
at least one processor; and
a non-transitory processor-readable memory device storing instructions that when executed by the at least one processor causes the at least one processor to perform operations including:
assigning each data party of the plurality of data parties with a corresponding namespace in an object store, wherein the corresponding namespace in the object store that the data party is assigned with is unique to the data party;
assigning a shared namespace in the object store, wherein the shared namespace is shared by the plurality of data parties;
storing one or more model parameters in the shared namespace in the object store;
triggering a round of federated learning by issuing one or more customized learning requests to one or more of the plurality of data parties, wherein each customized learning request issued to each data party of the one or more data parties comprises a link to the object store, a file name to use that is indicative of the round of federated learning, and information identifying both the corresponding namespace in the object store that the data party is assigned with and the shared namespace in the object store, and the customized learning request triggers the data party to fetch the one or more model parameters from the shared namespace in the object store, locally train a model based on training data owned by the data party and the one or more model parameters, and upload using the file name a local model resulting from the local training to the corresponding namespace in the object store that the data party is assigned with;
retrieving, from the object store, at least one local model uploaded to the object store by at least one data party of the plurality of data parties during the round of federated learning; and
aggregating the at least one local model retrieved from the object store to obtain a shared model.

13. The system of claim 12 , wherein the shared model is uploaded to the shared namespace in the object store.

14. The system of claim 13 , wherein the operations further comprise:
triggering a subsequent round of federated learning by issuing one or more subsequent customized learning requests to one or more of the plurality of data parties, wherein each subsequent customized learning request issued to each data party of the one or more data parties triggers the data party to retrieve the shared model from the shared namespace in the object store, locally train the shared model based on training data owned by the data party, and upload using a different file name that is indicative of the subsequent round of federated learning an updated version of a local model resulting from the local training to a corresponding namespace in the object store that the data party is assigned with.

15. The system of claim 12 , wherein triggering a round of federated learning comprises:
selecting a set of data parties; and
triggerin

Assignees

• IBM

Inventors

• Rajamoni Shashank
• Anwar Ali
• Zhou Yi
• Ludwig Heiko H
• Baracaldo Angel Nathalie

Classifications

• G06N20/00 Primary

  Machine learning · CPC title