Hybrid channel modeling for time series foundation models

Patent metadata

Field	Value
Publication number	US-2024394333-A1
Application number	US-202318200640-A
Country	US
Kind code	A1
Filing date	May 23, 2023
Priority date	May 23, 2023
Publication date	Nov 28, 2024
Grant date	—

Abstract

A method, system, and compute program product are configured to: receive a dataset comprising a multivariate time series that includes plural channels; generate an original forecast of the multivariate time series using a channel-independent backbone and a prediction head; and generate a revised forecast of the multivariate time series using a cross-channel reconciliation head with the original forecast, wherein the cross-channel reconciliation head generates the revised forecast based on correlations between the channels of the multivariate time series.

First claim

What is claimed is:

1 . A computer-implemented method of time series forecasting, comprising:
receiving, by a processor set, a dataset comprising a multivariate time series that includes plural channels; generating, by the processor set, an original forecast of the multivariate time series using a channel-independent backbone and a prediction head; and generating, by the processor set, a revised forecast of the multivariate time series using a cross-channel reconciliation head with the original forecast, wherein the cross-channel reconciliation head generates the revised forecast based on correlations between the channels of the multivariate time series.

2 . The computer-implemented method of claim 1 , wherein the cross-channel reconciliation head creates plural patches, wherein respective ones of the patches comprise: a respective forecast point of the original forecast; and a context-length number of surrounding forecast points of the original forecast before and after the respective forecast point of the original forecast.

3 . The computer-implemented method of claim 2 , wherein the cross-channel reconciliation head creates flattened patches by flattening the patches across the channels of the multivariate time series.

4 . The computer-implemented method of claim 3 , wherein the cross-channel reconciliation head generates respective revised forecast points of the revised forecast by applying a gated attention function to respective ones of the flattened patches.

5 . The computer-implemented method of claim 2 , wherein the cross-channel reconciliation head comprises a residual connection.

6 . The computer-implemented method of claim 1 , wherein the channel-independent backbone comprises a transformer-based backbone.

7 . The computer-implemented method of claim 1 , wherein the channel-independent backbone comprises a mixer-based backbone.

8 . The computer-implemented method of claim 7 , wherein the mixer-based backbone comprises a deep learning neural network model.

9 . The computer-implemented method of claim 1 , further comprising training the channel-independent backbone using multiple different datasets.

10 . The computer-implemented method of claim 1 , wherein the multivariate time series comprises sensor data from plural sensors in a system, and further comprising performing an action in the system based on the revised forecast of the multivariate time series.

11 . A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
receive a dataset comprising a multivariate time series that includes plural channels; generate an original forecast of the multivariate time series using a channel-independent backbone and a prediction head; and generate a revised forecast of the multivariate time series using a cross-channel reconciliation head with the original forecast, wherein the cross-channel reconciliation head generates the revised forecast based on correlations between the channels of the multivariate time series.

12 . The computer program product of claim 11 , wherein the cross-channel reconciliation head creates plural patches, wherein respective ones of the patches comprise: a respective forecast point of the original forecast; and a context-length number of surrounding forecast points of the original forecast before and after the respective forecast point of the original forecast.

13 . The computer program product of claim 12 , wherein the cross-channel reconciliation head creates flattened patches by flattening the patches across the channels of the multivariate time series.

14 . The computer program product of claim 13 , wherein the cross-channel reconciliation head generates respective revised forecast points of the revised forecast by applying a gated attention function to respective ones of the flattened patches.

15 . The computer program product of claim 12 , wherein the cross-channel reconciliation head comprises a residual connection.

16 . A system comprising:
a processor set, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to: receive a dataset comprising a multivariate time series that includes plural channels; generate an original forecast of the multivariate time series using a channel-independent backbone and a prediction head; and generate a revised forecast of the multivariate time series using a cross-channel reconciliation head with the original forecast, wherein the cross-channel reconciliation head generates the revised forecast based on correlations between the channels of the multivariate time series.

17 . The system of claim 16 , wherein the cross-channel reconciliation head creates plural patches, wherein respective ones of the patches comprise: a respective forecast point of the original forecast; and a context-length number of surrounding forecast points of the original forecast before and after the respective forecast point of the original forecast.

18 . The system of claim 17 , wherein the cross-channel reconciliation head creates flattened patches by flattening the patches across the channels of the multivariate time series.

19 . The system of claim 18 , wherein the cross-channel reconciliation head generates respective revised forecast points of the revised forecast by applying a gated attention function to respective ones of the flattened patches.

20 . The system of claim 17 , wherein the cross-channel reconciliation head comprises a residual connection.

Assignees

• IBM

Inventors

• Ekambaram Vijay
• Jati Arindam
• Nguyen Nam H
• Sinthong Phanwadee
• Kalagnanam Jayant R

Classifications

• G06F17/18 Primary

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title