System and Method For Multi-Task Learning Through Spatial Variable Embeddings

Patent metadata

Field	Value
Publication number	US-2022207362-A1
Application number	US-202117554097-A
Country	US
Kind code	A1
Filing date	Dec 17, 2021
Priority date	Dec 31, 2020
Publication date	Jun 30, 2022
Grant date	—

Abstract

A general prediction model is based on an observer traveling around a continuous space, measuring values at some locations, and predicting them at others. The observer is completely agnostic about any particular task being solved; it cares only about measurement locations and their values. A machine learning framework in which seemingly unrelated tasks can be solved by a single model is proposed, whereby input and output variables are embedded into a shared space. The approach is shown to (1) recover intuitive locations of variables in space and time, (2) exploit regularities across related datasets with completely disjoint input and output spaces, and (3) exploit regularities across seemingly unrelated tasks, outperforming task-specific single-task models and multi-task learning alternatives.

First claim

1 . A process, implemented in a computing environment, for training a single model across diverse tasks, comprising:
measuring tasks with disjoint input and output variable sets in a shared space; for each task,
encoding by a function f a value of each observed variable x i given its shared space location z i ;
aggregating encodings by elementwise addition; and decoding by a function g the aggregated encodings to predict y j at its location z j , wherein z i and z j are variable embeddings.

2 . The process according to claim 1 , wherein the encoding and decoding are conditioned on the variable embeddings via Feature-wise Linear Modulation (FiLM) layers.

3 . The process according to claim 1 , wherein function g is re-decomposed into a core, which is independent of output variable, and a decoder, g 2 , which is conditioned on output variable.

4 . The process according to claim 3 , wherein the single model is in the form of:
[ y j |x ]= g 2 ( g 1 (Σ i=1 n f ( x i ,z i )), z j ).

5 . The process according to claim 1 , wherein functions f and g are implemented as neural networks.

6 . The process according to claim 5 , wherein the neural networks are residual block networks. The process according to claim 1 , wherein the shared space is 2-Dimensional.

8 . At least one computer-readable medium storing instructions that, when executed by a computer, perform a process for training a single model across diverse tasks, the process comprising:
measuring tasks with disjoint input and output variable sets in a shared space; for each task,
encoding by a function f a value of each observed variable x i given its shared space location z i ;
aggregating encodings by elementwise addition; and
decoding by a function g the aggregated encodings to predict y j at its location z j , wherein z i and z j are variable embeddings.

9 . The at least one computer-readable medium of claim 8 , wherein the encoding and decoding are conditioned on the variable embeddings via Feature-wise Linear Modulation (FiLM) layers.

10 . The at least one computer-readable medium of claim 8 , wherein function g is re-decomposed into a core, g 1 , which is independent of output variable, and a decoder, g 2 , which is conditioned on output variable.

11 . The at least one computer-readable medium of claim 10 , wherein the single model is in the form of:
[ y j |x ]= g 2 ( g 1 (Σ i=1 n f ( x i ,z i )), z j ).

12 . The at least one computer-readable medium of claim 8 , wherein functions f and g are implemented as neural networks.

13 . The at least one computer-readable medium of claim 12 , wherein the neural networks are residual block networks.

14 . The computer-readable medium of claim 8 , wherein the shared space is 2-Dimensional.

15 . A single universal prediction model trained across diverse tasks in a shared space with disjoint input and output variable sets, the single universal prediction model comprising:
an encoder, f, which is conditioned on vector z i , for generating an encoder output for each task variable x i given its location in the shared space; an aggregator for aggregating the encoder outputs; a core, g 1 , which is independent of output variable; and a decoder, g 2 , which is conditioned on vector z j , for generating a prediction y j given its location in the shared space.

16 . The single universal prediction model of claim 15 , having the form of: [y j |x]=g 2 (g 1 (Σ i=1 n f(x i ,z i )), z j ).

17 . The single universal prediction model of claim 15 , wherein vector z i and z j are variable embeddings.

18 . The single universal prediction model of claim 15 , wherein functions f and g are implemented as neural networks.

19 . The single universal prediction model of claim 18 , wherein the neural networks are residual block networks.

20 . The single universal prediction model of claim 15 , wherein the encoder and decoder are conditioned on the variable embeddings via Feature-wise Linear Modulation (FiLM) layers.

Assignees

• Cognizant Tech Solutions U S Corporation

Inventors

• Meyerson Elliot
• Miikkulainen Risto

Classifications

• G06N20/10 Primary

  using kernel methods, e.g. support vector machines [SVM] · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title