Data processing method and apparatus

Patent metadata

Field	Value
Publication number	US-2018314944-A1
Application number	US-201816026784-A
Country	US
Kind code	A1
Filing date	Jul 3, 2018
Priority date	Oct 14, 2016
Publication date	Nov 1, 2018
Grant date	—

Abstract

A data processing method in a data processing device is provided. First to-be-processed data input into a neural network are obtained. Iterative training is performed on the neural network for a first preset number of times by using first target data in the first to-be-processed data, to obtain a seed model of the neural network. First newly added data generated after an elapse of time corresponding to the first time window is obtained, and the first newly added data and the first to-be-processed data are combined into second to-be-processed data. Iterative training is performed on the seed model for a second preset number of times by using second target data in the second to-be-processed data, to obtain a first incremental model of the neural network. A first preset area overlaps between the second time window and the first time window. The first incremental model online is published.

First claim

What is claimed is:

1 . A data processing method in a data processing device, comprising:
obtaining, by at least one processor of the data processing device, first to-be-processed data input into a neural network, the neural network being a to-be-updated model; performing, by the at least one processor of the data processing device, iterative training on the neural network for a first preset number of times by using first target data in the first to-be-processed data, to obtain a seed model of the neural network, the first target data being located in a first time window, and the seed model being an initialization model of the neural network; obtaining, by the at least one processor of the data processing device, first newly added data generated after an elapse of time corresponding to the first time window, and combining the first newly added data and the first to-be-processed data into second to-be-processed data; performing, by the at least one processor of the data processing device, iterative training on the seed model for a second preset number of times by using second target data in the second to-be-processed data, to obtain a first incremental model of the neural network, the second target data being located in a second time window, and a first preset area overlapping between the second time window and the first time window; and publishing, by the at least one processor of the data processing device, the first incremental model online.

2 . The method according to claim 1 , wherein a right boundary of the second time window conforms with current time.

3 . The method according to claim 1 , further comprising:
determining, by the at least one processor of the data processing device, a third time window according to first time and the second time window, a second preset area overlapping between the third time window and the second time window; obtaining, by the at least one processor of the data processing device, second newly added data generated after an elapse of time corresponding to the second time window, and combining the second newly added data and the second to-be-processed data into third to-be-processed data; performing, by the at least one processor of the data processing device, iterative training on the first incremental model for a third preset number of times by using third target data in the third to-be-processed data, to obtain a second incremental model of the neural network, wherein the third target data is located in the third time window; and publishing, by the at least one processor of the data processing device, the second incremental model online.

4 . The method according to claim 3 , wherein a right boundary of the third time window conforms with the first time.

5 . The method according to claim 3 , wherein the determining the third time window comprises:
determining whether there is indication information for pushing a new model, wherein the new model comprises the second incremental model; determining the first time as a right boundary of the third time window based on a result of determination that there is the indication information; sliding the second time window to the right boundary of the third time window, and pushing the right boundary of the third time window forward by a length of the third time window, to obtain a left boundary of the third time window; and determining the third time window according to the right boundary of the third time window and the left boundary of the third time window.

6 . The method according to claim 1 , further comprising:
determining, by the at least one processor of the data processing device, according to a preset period, whether failure data exists in the first incremental model, wherein the failure data is the first to-be-processed data stopped to be pushed; clearing, by the at least one processor of the data processing device, the failure data from the first incremental model to obtain an updated incremental model based on a result of determination, according to the preset period, that the failure data exists in the first incremental model; and publishing, by the at least one processor of the data processing device, the updated incremental model online.

7 . The method according to claim 6 , wherein the clearing comprises:
expanding the second time window by a preset multiple, to obtain a fourth time window; and obtaining the second to-be-processed data in the first incremental model, using the second to-be-processed data as the failure data, wherein the second to-be-processed data is not in the fourth time window, and clearing the failure data from the first incremental model to obtain the updated incremental model.

8 . The method according to claim 7 , further comprising:
determining, by the at least one processor of the data processing device, a fifth time window according to second time and the fourth time window, a third preset area overlapping between the fifth time window and the fourth time window; obtaining, by the at least one processor of the data processing device, third newly added data generated after an elapse of time corresponding to the fourth time window, and combining the third newly added data and the second to-be-processed data into fourth to-be-processed data; performing, by the at least one processor of the data processing device, iterative training on the updated incremental model for a fourth preset number of times by using fourth target data in the fourth to-be-processed data, to obtain a third incremental model of the neural network, wherein the fourth target data is located in the fifth time window; and publishing, by the at least one processor of the data processing device, the third incremental model online.

9 . The method according to claim 8 , wherein a right boundary of the fifth time window conforms with the second time.

10 . The method according to claim 1 , further comprising:
fitting historical data by using the first incremental model to obtain a fitting result, wherein the historical data is data obtained by processing previous to-be-processed data; and carrying the fitting result by using the first incremental model.

11 . The method according to claim 1 , wherein the performing the iterative training on the neural network comprises:
in a cold start state, performing random initialization on parameters of layers of the neural network, to obtain an initialization parameter, wherein the cold start state is a state when the neural network is processed for a first time; and performing the iterative training on the neural network for the first preset number of times by using the first target data and the initialization parameter, to obtain the seed model of the neural network.

12 . The method according to claim 11 , wherein the performing the random initialization comprises at least one of:
separately performing initialization on the parameters of the layers of the neural network based on a particular constant; performing even distribution random initialization on the parameters of the layers of the neural network; performing Gaussian distribution random initialization on the parameters of the layers of the neural network; or performing Xavier initialization on the parameters of the layers of the neural network.

13 . The method according to claim 1 , wherein the method is applied to a preset scenario, and the preset scenario comprises at least one of:
a click-through rate (CTR) pre-estimation scenario of a media file; a training scenario of an image recognition model; a training scenario of a voice recognition model; or a training scenario of a natural language understanding mode

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Li Yi
• Jin Xing
• Zhang Shubin
• Guo Zhimao
• Xue Wei

Classifications

• G06Q30/0241

  Advertisements · CPC title

• G06Q30/0242

  Determining effectiveness of advertisements · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06F17/16

  Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title