Systems and methods for non-stationary time-series forecasting

Patent metadata

Field	Value
Publication number	US-2023376746-A1
Application number	US-202217939085-A
Country	US
Kind code	A1
Filing date	Sep 7, 2022
Priority date	May 18, 2022
Publication date	Nov 23, 2023
Grant date	—

Abstract

Embodiments described herein provide a time-index model for forecasting time-series data. The architecture of the model takes a normalized time index as an input, uses a model, g_φ, to produce a vector representation of the time-index, and uses a “ridge regressor” which takes the vector representation and provides an estimated value. The model may be trained on a time-series dataset. The ridge regressor is trained for a given g_φ to reproduce a given lookback window. g_φ is trained over time-indexes in a horizon window, such that g_φ and the corresponding ridge regressor will accurately predict the data in the horizon window. Once g_φ is sufficiently trained, the ridge regressor can be updated based on that final g_φ over a lookback window comprising the time-indexes with the last known values. The final g_φ together with the updated ridge regressor can be given time-indexes past the known values, thereby providing forecasted values.

First claim

What is claimed is:

1 . A method of training a time series data forecasting model, the method comprising:
receiving a time-series data sequence including first time series data over a first lookback time window and second time series data over a first horizon time window following the lookback time window in time; generating, by a neural network parametrized by first parameters of a final layer and second parameters of other layers, first outputs based on an input of normalized time coordinates from the first lookback time window; updating the first parameters of the final layer based on a first training objective comparing the first time series data and the first outputs of the neural network while keeping the second parameters of the other layers frozen; generating, by the neural network parametrized with updated first parameters and the second parameters that have been frozen, second outputs based on an input of normalized time coordinates from the first horizon time window; and updating the second parameters based on a second training objective comparing the second time series data and second outputs of the neural network subject to the updated first parameters of the final layer.

2 . The method of claim 1 , further comprising:
generating, by the neural network, third outputs based on an input of normalized time coordinates from a second lookback time window of the time-series data sequence; and updating the first parameters of the final layer based on the first training objective comparing third time series data over the second lookback time window and the third outputs of the neural network; while keeping the second parameters of the other layers frozen;

3 . The method of claim 2 , further comprising:
generating, by the neural network, fourth outputs based on an input of normalized time coordinates from a second horizon time window; and updating the second parameters based on the second training objective comparing fourth time series data over the second horizon time window and the fourth outputs of the neural network subject to the updated first parameters of the final layer.

4 . The method of claim 1 , wherein the first training objective is computed by summing a cross entropy between the first time series data and the first outputs of the neural network over the first lookback time window.

5 . The method of claim 1 , wherein the second training objective is computed by summing a cross entropy between the second time series data and the second outputs of the neural network over the first horizon time window.

6 . The method of claim 1 , wherein the input of normalized time coordinates from the first lookback time window are modified by one or more sinusoid functions.

7 . The method of claim 1 , wherein the input of normalized time coordinates from the first lookback time window are modified by a concatenation of sinusoid functions.

8 . A system for training a time series data forecasting model, the system comprising:
a memory that stores the time series data forecasting model and a plurality of processor executable instructions; a communication interface that receives a time-series data sequence including first time series data over a first lookback time window and second time series data over a first horizon time window following the lookback time window in time; and one or more hardware processors that read and execute the plurality of processor-executable instructions from the memory to perform operations comprising:
generating, by a neural network parametrized by first parameters of a final layer and second parameters of other layers, first outputs based on an input of normalized time coordinates from the first lookback time window;
updating the first parameters of the final layer based on a first training objective comparing the first time series data and the first outputs of the neural network while keeping the second parameters of the other layers frozen;
generating, by the neural network parametrized with updated first parameters and the second parameters that have been frozen, second outputs based on an input of normalized time coordinates from the first horizon time window; and
updating the second parameters based on a second training objective comparing the second time series data and second outputs of the neural network subject to the updated first parameters of the final layer.

9 . The system of claim 8 , wherein the operations further comprise:
generating, by the neural network, third outputs based on an input of normalized time coordinates from a second lookback time window of the time-series data sequence; and updating the first parameters of the final layer based on the first training objective comparing third time series data over the second lookback time window and the third outputs of the neural network; while keeping the second parameters of the other layers frozen;

10 . The system of claim 9 , wherein the operations further comprise:
generating, by the neural network, fourth outputs based on an input of normalized time coordinates from a second horizon time window; and updating the second parameters based on the second training objective comparing fourth time series data over the second horizon time window and the fourth outputs of the neural network subject to the updated first parameters of the final layer.

11 . The system of claim 8 , wherein the first training objective is computed by summing a cross entropy between the first time series data and the first outputs of the neural network over the first lookback time window.

12 . The system of claim 8 , wherein the second training objective is computed by summing a cross entropy between the second time series data and the second outputs of the neural network over the first horizon time window.

13 . The system of claim 8 , wherein the input of normalized time coordinates from the first lookback time window are modified by one or more sinusoid functions.

14 . The system of claim 8 , wherein the input of normalized time coordinates from the first lookback time window are modified by a concatenation of sinusoid functions.

15 . A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:
receiving a time-series data sequence including first time series data over a first lookback time window and second time series data over a first horizon time window following the lookback time window in time; generating, by a neural network parametrized by first parameters of a final layer and second parameters of other layers, first outputs based on an input of normalized time coordinates from the first lookback time window; updating the first parameters of the final layer based on a first training objective comparing the first time series data and the first outputs of the neural network while keeping the second parameters of the other layers frozen; generating, by the neural network parametrized with updated first parameters and the second parameters that have been frozen, second outputs based on an input of normalized time coordinates from the first horizon time window; and updating the second parameters based on a second training objective comparing the second time series data and second outputs of the neural network subject to the updated first parameters of the final layer.

16 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
generating, by the neural network,

Assignees

• Salesforce Com Inc

Inventors

• Woo Gerald
• Liu Chenghao
• Sahoo Doyen
• Hoi Chu Hong

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0481

  Physics · mapped topic

• G06N3/048

  Activation functions · CPC title

• G06N3/0985 Primary

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title