Multi-level mixer masked autoencoder

Patent metadata

Field	Value
Publication number	US-2025371335-A1
Application number	US-202418731469-A
Country	US
Kind code	A1
Filing date	Jun 3, 2024
Priority date	Jun 3, 2024
Publication date	Dec 4, 2025
Grant date	—

Abstract

An approach for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model. The approach may involve expanding an input encoding from a time-series independent encoder and correlation encoding the expanded encodings. The approach may include compressing the correlation encodings. The approach may also include decoding the correlation encodings, based on a decoder head. Additionally, the approach may include determining the error of the decoded correlation encodings compared to the plurality of input features and updating one or more weights of the decoder head based on the error.

First claim

What is claimed is:

1 . A computer-implemented method for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model, the computer-implemented method comprising:
receiving, by the processor, a plurality of input features; expanding, by a processor, an input encoding from a time-series independent encoder; correlation encoding, by the processor, the expanded encodings, based at least in part on mixing the encodings through a least one of the following, channel mixing, spatial mixing, and channel mixing. compressing, by the processor, the correlation encodings; decoding, by the processor, the correlation encodings, based on a decoder head; determining, by the processor, the error of the decoded correlation encodings compared to the plurality of input features; and and updating, by the processor, one or more weights of the decoder head based on the error.

2 . The computer-implemented method of claim 1 , wherein the decoder head is comprised of a thin decoder with mixer architecture and a linear prediction head.

3 . The computer-implemented method of claim 1 , wherein encoding the expanded input features further comprises:
expanding the time-series independent encoding, the, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing.

4 . The computer-implemented method of claim 1 , further comprising:
inputting, by the processor, an active multi-variate time series into the updated decoder head; and predicting, by the processor, a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.

5 . The computer-implemented method of claim 4 , further wherein predicting the future state variable value comprises:
encoding, by the processor, the active multi-variate time series; expanding, by the processor, active multi-variate time series encodings from an initial size to n number of phases; decoding, by the processor, the expanded active multi-variate time series encodings, based on a decoder head; compressing, by the processor, the decoded active multi-variate time series encodings; and generating, by the processor, the state variable value from the decoded active multi-variate time series encodings based on a linear prediction head.

6 . The computer-implemented method of claim 4 , wherein the active multivariate time-series is associated with natural gas production system pressures.

7 . The computer-implemented method of claim 4 , wherein the active multivariate time-series is associated with chemical impurity output levels.

8 . A computer system for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model, the computer system comprising:
a processor; a memory in communication with the processor; one or more computer program instructions stored on the memory, when executed by the processor, cause the processor to perform one or more operations, the operations comprising: receive a plurality of input features; expand an input encoding from a time-series independent encoder; correlation encode the expanded encodings, based at least in part on mixing the encodings through a least one of the following, channel mixing, spatial mixing, and channel mixing; compress the correlation encodings; decode the correlation encodings, based on a decoder head; determine the error of the decoded correlation encodings compared to the plurality of input features; and and update one or more weights of the decoder head based on the error.

9 . The computer system of claim 8 , wherein the decoder head is comprised of a thin decoder with mixer architecture and a linear prediction head.

10 . The computer system of claim 8 , wherein encoding the expanded input features further comprises:
expanding the time-series independent encoding, the, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing.

11 . The computer system of claim 8 , further comprising:
input an active multi-variate time series into the updated decoder head; predict a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.

12 . The computer system of claim 11 , further wherein predicting the future state variable value comprises:
encode the active multi-variate time series; expand the active multi-variate time series encodings from an initial size to n number of phases; decode the expanded active multi-variate time series encodings, based on a decoder head; compress the decoded active multi-variate time series encodings; and generate the state variable value from the decoded active multi-variate time series encodings based on a linear prediction head.

13 . The computer system of claim 11 , wherein the active multivariate time-series is associated with natural gas production system pressures.

14 . The computer system of claim 11 , wherein the active multivariate time-series is associated with chemical impurity output levels.

15 . A computer program product for training a multi-level mixer masked autoencoder with channel mixing across group dimensions model, the computer program product comprising:
program instructions stored on a memory device, executable by a processor to perform one or more operations, where in the program instructions comprise instructions to: receive a plurality of input features; expand an input encoding from a time-series independent encoder; correlation encode the expanded encodings, based at least in part on mixing the encodings through a least one of the following, channel mixing, spatial mixing, and channel mixing; compress the correlation encodings; decode the correlation encodings, based on a decoder head; determine the error of the decoded correlation encodings compared to the plurality of input features; and and update one or more weights of the decoder head based on the error.

16 . The computer program product of claim 15 , wherein the decoder head is comprised of a thin decoder with mixer architecture and a linear prediction head.

17 . The computer program product of claim 15 , wherein encoding the expanded input features further comprises:
expanding the time-series independent encoding, the, based at least in part on mixing the masked encodings through at least one of the following: intra patch mixing and interpatch mixing.

18 . The computer program product of claim 15 , further comprising:
input an active multi-variate time series into the updated decoder head; predict a future state variable value for the active multi-variate time series, based at least in part on the updated decoder head.

19 . The computer program product of claim 18 , further wherein predicting the future state variable value comprises:
encode the active multi-variate time series; expand the active multi-variate time series encodings from an initial size to n number of phases; decode the expanded active multi-variate time series encodings, based on a decoder head; compress the decoded active multi-variate time series encodings; and generate the state variable value from the decoded active multi-variate time series encodings based on a linear prediction head.

20 . The computer program product of claim 18 , wherein the active multivariate time-series is associated with natural gas production system pressures.

Assignees

• IBM

Inventors

• Ekambaram Vijay
• Jati Arindam
• Nguyen Nam H
• Reddy Chandrasekhara K
• Kalagnanam Jayant R

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/0455 Primary

  Auto-encoder networks; Encoder-decoder networks · CPC title