Method for implementing model update and device thereof

Patent metadata

Field	Value
Publication number	US-12572850-B2
Application number	US-202217871084-A
Country	US
Kind code	B2
Filing date	Jul 22, 2022
Priority date	Jan 23, 2020
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A method for implementing model update is provided and used in a federated update framework. The method includes: a service device receives difference information uploaded by at least two clients. The service device performs calculation based on the difference information uploaded by the at least two clients, to obtain first difference consistency information, where the first difference consistency information indicates a consistency degree of the difference information uploaded by the at least two clients. The service device performs calculation based on the first difference consistency information, to obtain first training information, where the first training information is used to train a third model, and the third model is obtained by updating the first model by the service device based on the difference information uploaded by the at least two clients. The service device sends the first training information to the at least two clients.

First claim

What is claimed is:

1 . A method for implementing model update, comprising:
receiving, by a service device having at least one processor implemented either on a server or a cloud, difference information uploaded by at least two clients, wherein difference information uploaded by a client of the at least two clients is the difference information that is of a second model relative to a first model and obtained by the client through training based on the first model, the first model is received by the client from the service device, and the second model is obtained by the client through training based on the first model; performing, by the service device, calculation based on the difference information uploaded by the at least two clients, to obtain first difference consistency information, wherein the first difference consistency information indicates a consistency degree of the difference information uploaded by the at least two clients, wherein the service device obtains the first difference consistency information through calculation according to the following formula:

GCR
t

=

❘
"\[LeftBracketingBar]"

∑
i

g
i

❘
"\[RightBracketingBar]"

∑
i

❘
"\[LeftBracketingBar]"

g
i

❘
"\[RightBracketingBar]"

,

wherein
GCR t indicates the first difference consistency information, g i indicates difference information uploaded by an i th client of the at least two clients, and i is a positive integer greater than or equal to 1;
updating, by the service device, the first model based on the difference information uploaded by the at least two clients, to obtain a third model;
performing, by the service device, calculation based on the first difference consistency information, to obtain first training information, wherein the first training information is used to train the third model; and
sending, by the service device, the first training information and the third model to the at least two clients.

2 . The method for implementing model update according to claim 1 , wherein the first training information comprises at least one of a first quantity of training batches or a first learning rate, the first quantity of training batches is used by the client to determine a quantity of training batches, and the first learning rate is used by the client to determine a training learning rate.

3 . The method for implementing model update according to claim 2 , wherein, when the first training information comprises the first quantity of training batches, the first training information obtained by the service device through calculation based on the first difference consistency information meets the following condition:
when GCR t <GCR t-1 ,E t <E t-1 ; when GCR t =GCR t-1 ,E t =E t-1 ; or when GCR t >GCR t-1 ,E t >E t-1 , wherein GCR t-1 indicates second difference consistency information, the second difference consistency information is difference consistency information obtained by the service device through calculation before the service device obtains the first difference consistency information, GCR t indicates the first difference consistency information, E t-1 indicates a second quantity of training batches, the second quantity of training batches indicates a quantity of training batches delivered by the service device to the client before the service device obtains the first quantity of training batches, and E t indicates the first quantity of training batches.

4 . The method for implementing model update according to claim 3 , wherein performing, by the service device, the calculation based on the first difference consistency information to obtain the first training information further comprises:
performing, by the service device, calculation based on the first difference consistency information and first performance determining information, to obtain the first training information, wherein the first performance determining information indicates a weight of a performance determining factor used when the service device calculates the first training information.

5 . The method for implementing model update according to claim 4 , wherein the first performance determining information comprises at least one of an accuracy weight and a communication cost weight, the accuracy weight indicates a weight of model accuracy set when the service device calculates the first training information, and the communication cost weight indicates a weight of a communication resource used when the service device calculates the first training information.

6 . The method for implementing model update according to claim 5 , wherein, when the first training information comprises the first quantity of training batches, and the first determining performance information comprises the accuracy weight and the communication cost weight, the service device obtains the first quantity of training batches through calculation according to the following formula:

E
t

=

{

α
×

E

t
-
1

+
a

b

+

β
×

E

t
-
1

GCR
t

<

GCR

t
-
1

Assignees

• Huawei Tech Co Ltd

Inventors

• Si Xiaoyun
• Zhang Liang
• Hu Xinyu
• Wu Jun

Classifications

• G06N20/00 Primary

  Machine learning · CPC title

• G06N3/098 Primary

  Distributed learning, e.g. federated learning · CPC title

• G06N20/20 Primary

  Ensemble learning · CPC title