Developing machine-learning models

Patent metadata

Field	Value
Publication number	US-2023106985-A1
Application number	US-201917767688-A
Country	US
Kind code	A1
Filing date	Oct 9, 2019
Priority date	Oct 9, 2019
Publication date	Apr 6, 2023
Grant date	—

Abstract

A method (300) for using federated learning to develop a machine-learning model is disclosed. The method, performed by a management function, comprises developing a seed version of the machine-learning model using a machine-learning algorithm (310) and communicating the seed version of the machine-learning model to a plurality of distributed nodes, each of the plurality of distributed nodes being associated with a local data set (320). The method further comprises receiving, for each of the plurality of distributed nodes, a representation of distribution of data within the associated local data set (330), assigning each of the plurality of distributed nodes to a learning group on the basis of the received representations, wherein each learning group comprises a subset of the plurality of distributed nodes amongst which federated learning is to be performed (340), and obtaining at least one group version of the machine learning model for each learning group based on the node versions of the machine learning model developed by the distributed nodes in the learning group (350).

First claim

1 - 7 . (canceled)

8 . A method for using federated learning to develop a machine-learning model, the method, performed by a management function, comprising:
developing a seed version of the machine-learning model using a machine-learning algorithm; communicating the seed version of the machine-learning model to a plurality of distributed nodes, each of the plurality of distributed nodes being associated with a local data set; receiving, for each of the plurality of distributed nodes, a representation of distribution of data within the associated local data set; assigning each of the plurality of distributed nodes to a learning group on the basis of the received representations, wherein each learning group comprises a subset of the plurality of distributed nodes amongst which federated learning is to be performed; and obtaining at least one group version of the machine learning model for each learning group based on the node versions of the machine learning model developed by the distributed nodes in the learning group.

9 . The method of claim 8 , wherein obtaining at least one group version of the machine learning model for each learning group based on the node versions of the machine learning model developed by the distributed nodes in the learning group comprises receiving the at least one group version of the machine learning model for each learning group from a group management function of the respective learning group.

10 . The method of claim 8 , further comprising, for each learning group:
instantiating a group management function for the learning group; and instructing distributed nodes in the learning group to communicate representations of node version of the machine-learning model to the instantiated group management function.

11 . The method of claim 8 , further comprising developing an updated seed version of the machine-learning model based on the obtained group versions of the machine-learning model.

12 . The method of claim 8 , wherein the management function comprises a centralized management function, and a distributed management function, and wherein the distributed management function comprises a group management function for each learning group.

13 . The method of claim 12 , wherein the step of obtaining the at least one group version of the machine learning model for each learning group comprises:
generating the at least one group version of the machine learning model for each learning group at the distributed management function; and communicating the group versions of the machine learning model from the distributed management function to the centralized management function.

14 . The method of claim 13 , wherein the step of obtaining the at least one group version of the machine learning model for each learning group comprises, for each learning group:
obtaining, at a group management function for the group, a node version of the machine-learning model from each distributed node of the respective learning group, wherein the node version of the machine-learning model has been developed based on the seed version of the machine-learning model and a local data set associated with the respective distributed node, and using the machine-learning algorithm; combining, at the group management function, the obtained node versions of the machine-learning model to form a group version of the machine learning model for that learning group; and communicating, by the group management function, the group version of the machine learning model for that learning group to the centralized management function.

15 . The method of claim 12 , further comprising instructing the plurality of distributed nodes to communicate a representation of a node version of the machine-learning model, wherein the node version of the machine-learning model has been developed based on the seed version of the machine-learning model and a local data set associated the respective distributed node, and using the machine-learning algorithm.

16 . The method of claim 15 , wherein the step of instructing the each of the plurality of distributed nodes to communicate a representation of a node version of the machine-learning model comprises instructing the each of the plurality of distributed nodes to communicate a representation of a node version of the machine-learning model to a respective one of the group management functions in the distributed management function.

17 . The method of claim 8 , wherein
the representation of distribution of data within the local data set comprises any one of a Gaussian mixture model (GMM), a Euclidean distance, a L-2 distance, a maximum mean discrepancy (MMD), or a Jsensen-Renyi divergence, and the representation of distribution of data within the local data set further comprises a quantity of labels per predetermined category in the local data set.

18 . (canceled)

19 . The method of claim 8 , further comprising:
designing at least one hyper parameter for distributed nodes in a learning group using the representation of distribution of data within the local data set for distributed nodes assigned to the learning group; and communicating the designed at least one hyper parameter to distributed nodes assigned to the learning group.

20 . The method of claim 8 , wherein the plurality of distributed nodes are assigned to a learning group on the basis of the similarity of the received representations of distribution of data.

21 . The method of claim 8 , wherein assigning each of the plurality of distributed nodes to a learning group on the basis of the received representations comprises comparing received representations of distribution of data within the local data sets with a representation of distribution of data in a reference data set that is available to the management function.

22 . The method of claim 8 , wherein developing a seed version of the machine-learning model comprises combining representations of node versions of the machine-learning model received form distributed nodes.

23 . A method for using federated learning to develop a machine-learning model, the method, performed by a distributed node, comprising:
receiving a seed version of a machine-learning model, wherein the seed version of the machine-learning model has been developed using a machine-learning algorithm; generating a representation of distribution of data within a local data set associated with the distributed node; communicating the generated representation to a management function; developing a node version of the machine-learning model, based on the seed version of the machine-learning model and the associated local data set, and using the machine-learning algorithm; and communicating a representation of the node version of the machine-learning model to the management function.

24 . The method of claim 23 , wherein
the representation of distribution of data within the local data set comprises any one of a Gaussian mixture model (GMM), a Euclidean distance, a L-2 distance, a maximum mean discrepancy (MMD), or a Jsensen-Renyi divergence and the representation of distribution of data within the local data set further comprises a quantity of labels per predetermined category in the local data set.

25 . (canceled)

26 . The method of claim 23 , further comprising:
receiving form the management function at least one hyper parameter that is designed for a learning group to which the distributed node is assigned; and using the hyper parameter to develop a node version of the machine-learning

Assignees

• Ericsson Telefon Ab L M

Inventors

• Hu Wenfeng
• Vandikas Konstantinos
• Ickin Selim

Classifications

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/098 Primary

  Distributed learning, e.g. federated learning · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F8/36

  Software reuse · CPC title