Multi-level mixer masked autoencoder

Patent metadata

Field	Value
Publication number	US-2025371336-A1
Application number	US-202418731489-A
Country	US
Kind code	A1
Filing date	Jun 3, 2024
Priority date	Jun 3, 2024
Publication date	Dec 4, 2025
Grant date	—

Abstract

An approach for pretraining a time-series foundation model, based on masking one or more non-control variables associated with one or more channels of a time-series data set. Pretraining may include identifying one or more control variables from the plurality of process variates, masking all the process variates except the control variates and generate all the masked process variates except the control variables, based on the control variables The approach may further involve finetuning the time-series foundation model, where filtering may include based on filtering out the one or more non-control variables.

First claim

What is claimed is:

1 . A computer-implemented method for training time-series foundation model to manage different types of process variates, the computer-implemented method comprising:
pretraining, by a processor, a time-series foundation model, based on masking one or more non-control variables associated with one or more channels of a time-series data set; and finetuning, by the processor, the time-series foundation model, based on filtering out the one or more non-control variables.

2 . The computer-implemented method of claim 1 , wherein pretraining further comprises:
receiving, by the processor, a plurality of process variates from a plurality of channels, wherein each of the process variates are associated with a specific channel; identifying, by the processor, one or more control variables from the plurality of process variates; masking, by the processor, all the process variates except the control variates; and generating, by the processor, all the masked process variates except the control variables, based on the control variables.

3 . The computer-implemented method of claim 1 , wherein finetuning further comprises
identifying, by the processor, one or more control variables from the plurality of process variates; masking, by the processor, the plurality of process variates not identified as control variables; and generating, by the processor, embeddings for one or more channels, based on the control variables; filtering, by the processor, the generated embeddings of one or more channels which are not part of the control variables, and are not a part of a forecast variable and a conditional variable, based on a decoder architecture; generating, by the processor, a dependency graph based on the filtered embeddings; and identifying, by the processor, one or more target variables based on the dependency graph.

4 . The computer implemented method of claim 3 , further comprising:
filtering, by the processor, the embeddings associated with the forecast channels; determining, by the processor, the error against the time-series data set; and updating, by the processor, one or more weights associated with a decoder model for predicting state variables of one or more channels of the time series data set.

5 . The computer-implemented method of claim 4 , further comprising:
inputting, by the processor, an active multi-variate time series into the updated decoder head; and predicting, by the processor, a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.

6 . The computer-implemented method of claim 4 , wherein the active multivariate time-series is associated with natural gas production system pressures.

7 . The computer-implemented method of claim 4 , wherein the active multivariate time-series is associated with forecasting temperature and pressure of a natural gas production system.

8 . A computer system for training a time-series foundation model to manage different types of process variates, the computer system comprising:
a processor; a memory in communication with the processor; one or more computer program instructions stored on the memory, when executed by the processor, cause the processor to perform one or more operations, the operations comprising: pretrain a time-series foundation model, based on masking one or more non-control variables associated with one or more channels of a time-series data set; and finetune the time-series foundation model, based on filtering out the one or more non-control variables.

9 . The computer system of claim 8 , wherein pretraining further comprises operations to:
receive a plurality of process variates from a plurality of channels, wherein each of the process variates are associated with a specific channel; identify one or more control variables from the plurality of process variates; mask all the process variates except the control variates; and generate all the masked process variates except the control variables, based on the control variables.

10 . The computer system of claim 8 , wherein finetuning further comprises operations to identify one or more control variables from the plurality of process variates;
mask the plurality of process variates not identified as control variables; generate embeddings for one or more channels, based on the control variables; filter the generated embeddings of one or more channels which are not part of the control variables, and are not a part of a forecast variable and a conditional variable, based on a decoder architecture; generate a dependency graph based on the filtered embeddings; and identify one or more target variables based on the dependency graph.

11 . The computer system of claim 8 , further comprising operations to:
filter the embeddings associated with the forecast channels; determine the error against the time-series data set; and updating, by the processor, one or more weights associated with a decoder model for predicting state variables of one or more channels of the time series data set.

12 . The computer system of claim 11 , further comprising operations to:
input an active multi-variate time series into the updated decoder head; and predict a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.

13 . The computer system of claim 11 , wherein the active multivariate time-series is associated with natural gas production system pressures.

14 . The computer system of claim 11 , wherein the active multivariate time-series is associated with forecasting temperature and pressure of a natural gas production system.

15 . A computer program product for training a time-series foundation model to manage different types of process variates, the computer program product comprising:
program instructions stored on a memory device, executable by a processor to perform one or more operations, where in the program instructions comprise instructions to:
pretrain a time-series foundation model, based on masking one or more non-control variables associated with one or more channels of a time-series data set; and
finetune the time-series foundation model, based on filtering out the one or more non-control variables.

16 . The computer program product of claim 15 , wherein pretraining further comprises, program instructions to:
receive a plurality of process variates from a plurality of channels, wherein each of the process variates are associated with a specific channel; identify one or more control variables from the plurality of process variates; mask all the process variates except the control variates; and generate all the masked process variates except the control variables, based on the control variables.

17 . The computer program product of claim 15 , wherein finetuning further comprises program instructions to:
identify one or more control variables from the plurality of process variates; mask the plurality of process variates not identified as control variables; generate embeddings for one or more channels, based on the control variables; filter the generated embeddings of one or more channels which are not part of the control variables, and are not a part of a forecast variable and a conditional variable, based on a decoder architecture; generate a dependency graph based on the filtered embeddings; and identify one or more target variables based on the dependency graph.

18 . The computer program product of claim 15 ,

Assignees

• IBM

Inventors

• Reddy Chandrasekhara K
• Ekambaram Vijay
• Jati Arindam
• Nguyen Nam H
• Kalagnanam Jayant R
• Sarpatwar Kanthi
• Sinthong Phanwadee

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/08 Primary

  Learning methods · CPC title