Communication efficient machine learning of data across multiple sites

Patent metadata

Field	Value
Publication number	US-11699080-B2
Application number	US-201816131150-A
Country	US
Kind code	B2
Filing date	Sep 14, 2018
Priority date	Sep 14, 2018
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

In one embodiment, a service receives machine learning-based generative models from a plurality of distributed sites. Each generative model is trained locally at a site using unlabeled data observed at that site to generate synthetic unlabeled data that mimics the unlabeled data used to train the generative model. The service receives, from each of the distributed sites, a subset of labeled data observed at that site. The service uses the generative models to generate synthetic unlabeled data. The service trains a global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models.

First claim

What is claimed is:

1. A method comprising:
receiving, at a service, machine learning-based generative models from a plurality of distributed sites, wherein each generative model is trained locally at a site using unlabeled data observed at that site to generate synthetic unlabeled data that mimics the unlabeled data used to train the generative model;
receiving, at the service and from each of the distributed sites, a subset of labeled data observed at that site;
using, by the service, the generative models to generate synthetic unlabeled data; and
training, by the service, a global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models.

2. The method as in claim 1 , wherein each generative model generates an amount of synthetic unlabeled data in proportion to the unlabeled data observed at the corresponding site at which the model was trained.

3. The method as in claim 1 , wherein the generative models are trained using labeled data having one or more labels that differ from a label that the global machine learning-based model uses.

4. The method as in claim 1 , further comprising:
providing, by the service, the global machine learning-based model to one or more of the distributed sites, wherein the one or more distributed sites use the global machine learning-based model to select which labeled data is to be sent to the service.

5. The method as in claim 1 , further comprising:
receiving, at the service, an unlabeled data sample from a particular one of the sites;
using, by the service, the global machine learning-based model to apply a label to the received data sample; and
providing, by the service, an indication of the label applied to the received data sample to the particular site from which the data sample was received.

6. The method as in claim 1 , wherein the labeled and unlabeled data observed at the sites comprise image data captured by cameras deployed at sites or network telemetry data.

7. The method as in claim 1 , further comprising:
requesting, by the service, additional labeled data from one or more of the sites, based on a loss function associated with the machine learning-based model.

8. The method as in claim 1 , wherein the generative models are generative adversarial network (GAN) models.

9. The method as in claim 1 , wherein training the global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models comprises:
mapping the received subsets of labeled data to a feature space, wherein the global machine learning-based model is trained using semi-supervised learning within the feature space.

10. The method as in claim 1 , wherein training the global machine learning-based model comprises using an autoencoder on the synthetic data to generate features for the global machine learning-based model.

11. An apparatus, comprising:
one or more network interfaces to communicate with a network;
a processor coupled to the network interfaces and configured to execute one or more processes; and
a memory configured to store a process executable by the processor, the process when executed configured to:
receive machine learning-based generative models from a plurality of distributed sites, wherein each generative model is trained locally at a site using unlabeled data observed at that site to generate synthetic unlabeled data that mimics the unlabeled data used to train the generative model;
receive, from each of the distributed sites, a subset of labeled data observed at that site;
use the generative models to generate synthetic unlabeled data; and
train a global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models.

12. The apparatus as in claim 11 , wherein each generative model generates an amount of synthetic unlabeled data in proportion to the unlabeled data observed at the corresponding site at which the model was trained.

13. The apparatus as in claim 11 , wherein the distributed sites each comprise a local area network, and wherein the apparatus provides a cloud-based service to the local area networks.

14. The apparatus as in claim 11 , wherein the process when executed is further configured to:
provide the global machine learning-based model to one or more of the distributed sites, wherein the one or more distributed sites use the global machine learning-based model to select which labeled data is to be sent to the service.

15. The apparatus as in claim 11 , wherein the process when executed is further configured to:
receive an unlabeled data sample from a particular one of the sites;
use the global machine learning-based model to apply a label to the received data sample; and
provide an indication of the label applied to the received data sample to the particular site from which the data sample was received.

16. The apparatus as in claim 11 , wherein the subset of labeled data observed at the site are selected randomly for sending to the apparatus.

17. The apparatus as in claim 11 , wherein the generative models are trained using labeled data having one or more labels that differ from a label that the global machine learning-based model uses.

18. The apparatus as in claim 11 , wherein the apparatus trains the global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models by:
mapping the received subsets of labeled data to a feature space, wherein the global machine learning-based model is trained using semi-supervised learning within the feature space.

19. A tangible, non-transitory, computer-readable medium storing program instructions that cause a service to execute a process comprising:
receiving, at the service, machine learning-based generative models from a plurality of distributed sites, wherein each generative model is trained locally at a site using unlabeled data observed at that site to generate synthetic unlabeled data that mimics the unlabeled data used to train the generative model;
receiving, at the service and from each of the distributed sites, a subset of labeled data observed at that site;
using, by the service, the generative models to generate synthetic unlabeled data; and
training, by the service, a global machine learning-based model using the received subsets of labeled data received from the distributed sites and the synthetic unlabeled data generated by the generative models.

20. The computer-readable medium as in claim 19 , wherein the process further comprises:
receiving, at the service, an unlabeled data sample from a particular one of the sites;
using, by the service, the global machine learning-based model to apply a label to the received data sample; and
providing, by the service, an indication of the label applied to the received data sample to the particular site from which the data sample was received.

Assignees

• Cisco Tech Inc

Inventors

• Zhu Xiaoqing
• Wang Yaqi
• Tan Dan
• Liston Rob
• Nikkhah Mehdi

Classifications

• G06N3/088 Primary

  Non-supervised learning, e.g. competitive learning · CPC title

• G06F18/253

  of extracted features · CPC title

• G06N20/00

  Machine learning · CPC title

• H04Q9/00

  Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom · CPC title

• G06F18/2155 Primary

  characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling · CPC title