Method and device for training a model based on federated learning

Patent metadata

Field	Value
Publication number	US-12056582-B2
Application number	US-202016921207-A
Country	US
Kind code	B2
Filing date	Jul 6, 2020
Priority date	Jul 3, 2019
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

A method and device for training a model based on federated learning are provided. The method includes: receiving a second original independent variable calculated value from a second data provider device; the second original independent variable calculated value being calculated by the second data provider device according to a second original independent variable and a second model parameter; calculating a dependent variable estimation value according to a first model parameter initial value of a first provider device, a first original independent variable of the first data provider device, and the second original independent variable calculated value; calculating a difference between a dependent variable of the first data provider device and the dependent variable estimation value; calculating a gradient of a loss function with respect to a first model parameter, according to the difference; and updating the first model parameter according to the gradient of the loss function with respect to the first model parameter.

First claim

What is claimed is:

1. A method for training a model based on federated learning, executed by a first data provider device, comprising:
receiving, from a second data provider device, a second original independent variable calculated value, the second original independent variable calculated value being calculated by the second data provider device according to a second original independent variable and a second model parameter;
calculating a dependent variable estimation value according to:
a first model parameter initial value of the first data provider device,
a first original independent variable of the first data provider device, and
the second original independent variable calculated value;

calculating a difference between a dependent variable of the first data provider device and the dependent variable estimation value;
calculating a gradient of a loss function with respect to a first model parameter, according to the difference; and
updating the first model parameter according to the gradient of the loss function with respect to the first model parameter,
wherein each of the second original independent variable calculated value, the dependent variable estimation value, the difference between the dependent variable of the first data provider device and the dependent variable estimation value, and the gradient of the loss function with respect to the first model parameter is not encrypted data,
and wherein the method further comprises:
prior to the receiving the second original independent variable calculated value from the second data provider device:
generating a pair of keys; and
sending, to the second data provider device, a public key in the pair of keys; and

after the calculating the difference between the dependent variable of the first data provider device and the dependent variable estimation value:
encrypting the difference by using a private key in the pair of keys to obtain an encrypted difference, and sending the encrypted difference to the second data provider device;
receiving an encrypted gradient of a loss function with respect to a second model parameter from the second data provider device, wherein the encrypted gradient of the loss function with respect to the second model parameter is obtained by the second data provider device performing a calculation on a random number and the encrypted difference with the public key;
decrypting, by using the private key in the pair of keys, the encrypted gradient of the loss function with respect to the second model parameter, to obtain a sum of the random number and the gradient of the loss function with respect to the second model parameter; and
sending the sum of the random number and the gradient of the loss function with respect to the second model parameter to the second data provider device.

2. The method according to claim 1 , wherein the calculating the dependent variable estimation value according to the first model parameter initial value of the first data provider device, the first original independent variable of the first data provider device, and the second original independent variable calculated value comprises:
obtaining a first original independent variable calculated value according to the first model parameter initial value of the first data provider device and the first original independent variable of the first data provider device;
adding the first original independent variable calculated value and the second original independent variable calculated value, to obtain an independent variable; and
obtaining the dependent variable estimation value by calculating a sigmoid function value of the independent variable.

3. The method according to claim 2 , wherein the calculating the gradient of the loss function with respect to the first model parameter, according to the difference comprises:
calculating the gradient of the loss function with respect to the first model parameter according to a following formula:

∂
L

∂

Θ
A

=

-

1
n

⁢

∑

i
=
1

n

⁢

(

y
i

-

h
Θ

⁡

(

x

i
⁢

⁢
1

)

)

⁢

x
i
A

;

wherein n is a number of dependent variables, y i is an original dependent variable, x i A is the first original independent variable, A stands for the first data provider device, h Σ (x i1 ) is the dependent variable estimation value, and X i1 is the independent variable.

4. The method according to claim 3 , wherein the first data provider device is provided with a parameter server and multiple working nodes.

5. The method according to claim 2 , wherein the first data provider device is provided with a parameter server and multiple working nodes.

6. The method according to claim 1 , wherein the first data provider device is provided with a parameter server and multiple working nodes.

7. A non-transitory computer-readable storage medium comprising computer programs stored thereon, wherein the programs, when executed by a processor, cause the processor to implement the method according to claim 1 .

8. A method for training a model based on federated learning, executed by a second data provider, comprising:
obtaining a second original independent variable calculated value according to a second model parameter and a second original independent variable of a second data provider device;
sending the second original independent variable calculated value to a first data provider device;
receiving, from the first data provider device, an encrypted difference between a dependent variable and a dependent variable estimation value; wherein the encrypted difference is obtained by the first data provider device encrypting a difference with a private key; th

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Ren Bing
• Yang Shengwen
• Zhou Xuhui

Classifications

• G06F2221/2107

  File encryption · CPC title

• G06F21/602 Primary

  Providing cryptographic facilities or services · CPC title

• H04L9/0825

  using asymmetric-key encryption or public key infrastructure [PKI], e.g. key signature or public key certificates · CPC title

• H04L9/3073

  involving pairings, e.g. identity based encryption [IBE], bilinear mappings or bilinear pairings, e.g. Weil or Tate pairing · CPC title

• H04L9/008 Primary

  involving homomorphic encryption · CPC title