Distributed machine learning for improved privacy

Patent metadata

Field	Value
Publication number	US-11809972-B2
Application number	US-202217726338-A
Country	US
Kind code	B2
Filing date	Apr 21, 2022
Priority date	Oct 11, 2017
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Methods, computer readable media, devices, and systems provide for distributed machine learning. In one aspect, a method of training a model is disclosed. The method includes receiving, by a client device, from one or more servers, an intermediate model, training, by the client device, the intermediate model based on private data, and transmitting, by the client device, to the one or more servers, the trained intermediate model.

First claim

What is claimed is:

1. A method comprising:
receiving, by a device from a server, an intermediate model;
accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device;
storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models;
receiving, from the server, model data representing a state of the intermediate model;
training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and
transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device.

2. The method of claim 1 , the intermediate model being trained without training a second model, further comprising deleting the training data after training the intermediate model.

3. The method of claim 1 , further comprising adding random noise to the trained intermediate model prior to transmitting the trained intermediate model to the server.

4. The method of claim 1 , wherein the server integrates the trained intermediate model received from the device with a second model.

5. The method of claim 1 , wherein the server integrates the trained intermediate model received from the device with other model data.

6. The method of claim 5 , wherein the other model data is received by the server from a second device, the second device generating the other model data using additional data that is private to the second device.

7. The method of claim 1 , further comprising:
transmitting, to the server, a second message indicating data identifiers of the training data stored on the device, the server selecting the intermediate model based on the data identifiers indicated in the second message.

8. The method of claim 1 , further comprising:
transmitting, to the server, a second message indicating a model to fit using the training data that has been collected by the device, the server selecting the intermediate model based on the model indicated in the second message.

9. The method of claim 1 , further comprising:
determining of a list of data identifiers for the intermediate model based on the mapping.

10. The method of claim 1 , further comprising:
receiving, from the server, data identifiers identifying data for training the intermediate model, wherein training the intermediate model comprises searching a data store to identify data based on the data identifiers, and providing the identified data to the intermediate model to train the intermediate model.

11. The method of claim 10 , wherein the identified data comprises personally identifiable information (PII), and wherein the PII is not present in the trained intermediate model.

12. The method of claim 1 , where the intermediate model is trained by performing a regular linear regression, logistic regression, neural network training, or Bayesian model training, based on the accessed training data.

13. A system comprising:
a memory storing instructions; and
one or more processors configured to execute the instructions stored in the memory for performing operations comprising:
receiving, by a device from a server, an intermediate model;
accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device;
storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models;
receiving, from the server, model data representing a state of the intermediate model;
training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and
transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device.

14. The system of claim 13 , wherein the intermediate model is trained without training a second model, the operations further comprising deleting the training data after training the intermediate model.

15. The system of claim 13 , the operations further comprising adding random noise to the trained intermediate model prior to transmitting the trained intermediate model to the server.

16. The system of claim 13 , wherein the server integrates the trained intermediate model received from the device with a second model.

17. The system of claim 13 , wherein the server integrates the trained intermediate model received from the device with other model data.

18. The system of claim 17 , wherein the other model data is received by the server from a second device, the second device generating the other model data using additional data that is private to the second device.

19. The system of claim 13 , the operations further comprising:
transmitting, to the server, a second message indicating data identifiers of the training data stored on the device, the server selecting the intermediate model based on the data identifiers indicated in the second message.

20. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, configure the one or more processors to perform operations comprising:
receiving, by a device from a server, an intermediate model;
accessing, by the device, training data based on a first message received from the server that indicates a list of active data identifiers, the training data comprising data that is private to the device;
storing a mapping between a plurality of models and corresponding lists of one or more input parameters used for training each of the plurality of models;
receiving, from the server, model data representing a state of the intermediate model;
training, by the device, the intermediate model based on the accessed training data and based on a determination that the device comprises a type of data associated with the intermediate model; and
transmitting, by the device, the trained intermediate model to the server without sharing the data that is private to the device.

Assignees

• Snap Inc

Inventors

• Pihur Vasyl
• Sankuratripati Subhash
• Huang Dachuan
• Marcedone Antonio
• Liu Frederick
• Zeng Ruogu

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• H04L51/18

  Commands or executable codes · CPC title

• H04L51/52

  for supporting social networking services · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title