Adaptive asynchronous federated learning

Patent metadata

Field	Value
Publication number	US-11574254-B2
Application number	US-202016861284-A
Country	US
Kind code	B2
Filing date	Apr 29, 2020
Priority date	Apr 29, 2020
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

Techniques for adaptive asynchronous federated learning are described herein. An aspect includes providing a first version of a global parameter to a first client and a second client. Another aspect includes receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client. Another aspect includes determining whether the first version of the global parameter matches a most recent version of the global parameter. Another aspect includes, based on determining that the first version of the global parameter does not match the most recent version of the global parameter, selecting a version of the global parameter. Another aspect includes aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

First claim

What is claimed is:

1. A computer-implemented method comprising:
providing, by a processor, a first version of a global parameter to a first client and a second client;
receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client;
determining whether the first version of the global parameter matches a most recent version of the global parameter;
based on determining that the first version of the global parameter does not match the most recent version of the global parameter, and based on determining a distance between the first gradient and a second gradient of the second client, selecting a version of the global parameter; and
aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

2. The method of claim 1 , wherein the second gradient is determined by the second client based on the first version of the global parameter and a respective second local dataset of the second client.

3. The method of claim 2 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being less than a threshold, selecting the most recent version of the global parameter.

4. The method of claim 2 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being greater than a threshold, selecting an earlier version of the global parameter.

5. The method of claim 4 further comprising:
aggregating the first gradient with multiple versions of the global parameter to determine multiple updated versions of the global parameter; and
selecting, based on a validation dataset, a best version of the global parameter from the multiple updated versions of the global parameter.

6. The method of claim 4 further comprising, based on selecting the earlier version of the global parameter, notifying the second client to reduce an update frequency of the second client.

7. The method of claim 1 further comprising providing the updated version of the global parameter to the first client and the second client.

8. A system comprising:
a memory having computer readable instructions; and
one or more processors for executing the computer readable instructions, the computer readable instructions controlling the one or more processors to perform operations comprising:
providing a first version of a global parameter to a first client and a second client;
receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client;
determining whether the first version of the global parameter matches a most recent version of the global parameter;
based on determining that the first version of the global parameter does not match the most recent version of the global parameter, and based on determining a distance between the first gradient and a second gradient of the second client, selecting a version of the global parameter; and
aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

9. The system of claim 8 , wherein the second gradient is determined by the second client based on the first version of the global parameter and a respective second local dataset of the second client.

10. The system of claim 9 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being less than a threshold, selecting the most recent version of the global parameter.

11. The system of claim 9 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being greater than a threshold, selecting an earlier version of the global parameter.

12. The system of claim 11 further comprising:
aggregating the first gradient with multiple versions of the global parameter to determine multiple updated versions of the global parameter; and
selecting, based on a validation dataset, a best version of the global parameter from the multiple updated versions of the global parameter.

13. The system of claim 11 further comprising, based on selecting the earlier version of the global parameter, notifying the second client to reduce an update frequency of the second client.

14. The system of claim 8 , further comprising providing the updated version of the global parameter to the first client and the second client.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform operations comprising:
providing a first version of a global parameter to a first client and a second client;
receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client;
determining whether the first version of the global parameter matches a most recent version of the global parameter;
based on determining that the first version of the global parameter does not match the most recent version of the global parameter, and based on determining a distance between the first gradient and a second gradient of the second client, selecting a version of the global parameter; and
aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

16. The computer program product of claim 15 , wherein the second gradient is determined by the second client based on the first version of the global parameter and a respective second local dataset of the second client.

17. The computer program product of claim 16 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being less than a threshold, selecting the most recent version of the global parameter.

18. The computer program product of claim 16 , wherein selecting the version of the global parameter based on the determined distance comprises, based on the determined distance being greater than a threshold, selecting an earlier version of the global parameter.

19. The computer program product of claim 18 further comprising:
aggregating the first gradient with multiple versions of the global parameter to determine multiple updated versions of the global parameter; and
selecting, based on a validation dataset, a best version of the global parameter from the multiple updated versions of the global parameter.

20. The computer program product of claim 18 further comprising, based on selecting the earlier version of the global parameter:
notifying the second client to reduce an update frequency of the second client.

Assignees

• IBM

Inventors

• Wang Shiqiang
• Tuor Tiffany
• Liu Changchang
• Le Thai Franck

Classifications

• G06N20/20 Primary

  Ensemble learning · CPC title