Systems and methods for generating contextual table embeddings for tabular data

Patent metadata

Field	Value
Publication number	US-2024242024-A1
Application number	US-202318179767-A
Country	US
Kind code	A1
Filing date	Mar 7, 2023
Priority date	Jan 13, 2023
Publication date	Jul 18, 2024
Grant date	—

Abstract

Systems and methods for generating contextual table embeddings for tabular data are disclosed. In one embodiment, a method may include: receiving, by a table embedding computer program, an input table comprising a plurality of cells; separating, by the table embedding computer program, the cells in the input table by data type, wherein the data type comprises a text data type or a numeric data type; embedding, by the table embedding computer program, the data type in each cell of the input table; enhancing, by the table embedding computer program, the cells of the input table based on a position and/or the data type; generating, by the table embedding computer program, contextual embeddings for the input table using an encoder of a table transformer; and generating, by the table embedding computer program, a table summary for the contextual embeddings using a decoder for the table transformer.

First claim

1 . A method for generating contextual embeddings for tabular data, comprising:
receiving, by a table embedding computer program, an input table comprising a plurality of cells; separating, by the table embedding computer program, the cells in the input table by data type, wherein the data type comprises a text data type or a numeric data type; embedding, by the table embedding computer program, the data type in each cell of the input table; enhancing, by the table embedding computer program, the cells of the input table based on a position and/or the data type; generating, by the table embedding computer program, contextual embeddings for the input table using an encoder of a table transformer; and generating, by the table embedding computer program, a table summary for the contextual embeddings using a decoder for the table transformer.

2 . The method of claim 1 , wherein semantic representations of text present in the cells of the text data type are embedded in the cells.

3 . The method of claim 1 , wherein embedding vectors are embedded into the cells using linear projection.

4 . The method of claim 1 , wherein the computer program enhances each of the cells with a Fourier encoding of its position in the input table.

5 . The method of claim 1 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

6 . The method of claim 1 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.

7 . A system, comprising:
a data repository comprising a plurality of tables each comprising a plurality of cells; an electronic device executing a table embedding computer program that receives an input table out of the plurality of tables from the data repository, separates the cells in the input table by data type, wherein the data type comprises a text data type or a numeric data type, embeds the data type in each cell of the input table, enhances the cells of the input table based on a position and/or the data type, generates contextual embeddings for the input table using an encoder of a table transformer, and generates a table summary for the contextual embeddings using a decoder for the table transformer; and a downstream system that receives the contextual embeddings from the table embedding computer program.

8 . The system of claim 7 , wherein semantic representations of text present in the cells of the text data type are embedded in the cells.

9 . The system of claim 7 , wherein embedding vectors are embedded into the cells using linear projection.

10 . The system of claim 7 , wherein the computer program enhances each of the cells with a Fourier encoding of its position in the input table.

11 . The system of claim 7 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

12 . The system of claim 7 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.

13 . A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:
receiving an input table comprising a plurality of cells; separating the cells in the input table by data type, wherein the data type comprises a text data type or a numeric data type; embedding the data type in each cell of the input table; enhancing the cells of the input table based on a position and/or the data type; generating contextual embeddings for the input table using an encoder of a table transformer; and generating a table summary for the contextual embeddings using a decoder for the table transformer.

14 . The non-transitory computer readable storage medium of claim 13 , wherein semantic representations of text present in the cells of the text data type are embedded in the cells.

15 . The non-transitory computer readable storage medium of claim 13 , wherein embedding vectors are embedded into the cells using linear projection.

16 . The non-transitory computer readable storage medium of claim 13 , wherein the computer program enhances each of the cells with a Fourier encoding of its position in the input table.

17 . The non-transitory computer readable storage medium of claim 13 , wherein the encoder of the table transformer is trained to generate the contextual embeddings using an attention mechanism.

18 . The non-transitory computer readable storage medium of claim 13 , wherein the table summary comprises a sequence of text, and the decoder is trained to generate the sequence of text.

Assignees

• Jpmorgan Chase Bank Na

Inventors

• Moon Sae Young
• Tao Lili
• Moran Sean
• Gourgoulias Kostis

Classifications

• G06F40/177

  of tables; using ruled lines · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F40/183

  Tabulation, i.e. one-dimensional [1D] positioning · CPC title

• G06F40/126

  Character encoding · CPC title

• G06F40/18 Primary

  of spreadsheets (form-filling G06F40/174) · CPC title