Techniques for data type detection with learned metadata

Patent metadata

Field	Value
Publication number	US-12282829-B2
Application number	US-202117412034-A
Country	US
Kind code	B2
Filing date	Aug 25, 2021
Priority date	Aug 25, 2021
Publication date	Apr 22, 2025
Grant date	Apr 22, 2025

Abstract

Various embodiments are generally directed to techniques for creating and utilizing multidimensional embedding spaces for data objects, such as to condition the data for input to a neural network, for instance. Some embodiments are particularly directed to detecting data types for structured data based on learned metadata. In many embodiments, an embedding space for a set of data objects may be customized with a set of dimensions that correspond to various characteristics of the set of data objects. For example, a set of data objects may correspond to a table with each data object corresponding to a data entry in the table. In such examples, correlations between different columns and/or within a column of data may be identified and utilized as metadata to improve classifications by the neural network.

First claim

What is claimed is:

1. An apparatus, the apparatus comprising:
a processor; and
memory comprising instructions that when executed by the processor cause the processor to:
identify a set of data objects, the set of data objects comprising an array of values, wherein each data object in the set of data objects comprises a column value, a row value, and a data value;
determine a first group of data objects in the set of data objects, the first group of data objects corresponding to a first column in the array of values;
determine a second group of data objects in the set of data objects, the second group of data objects corresponding to a second column in the array of values;
concatenate data values from the first group of data objects with the data values from the second group of data objects in a row-wise manner to produce a concatenated group of data values;
determine at least one of a set of a plurality of embedding space parameters based on the concatenated group of data values, wherein the set of embedding space parameters define an embedding space comprising a plurality of dimensions; and
generate a set of object vectors, the set of object vectors comprising an object vector for each data object in the set of data objects, each object vector in the set of object vectors to include a set of dimension values and each dimension value in the set of dimension values to correspond to one of the plurality of dimensions in the embedding space, wherein a respective object vector for a respective data object is generated based on a respective column value of the respective data object, a respective row value of the respective data object, and the set of embedding space parameters.

2. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:
analyze each data value in the first group of data objects to determine one or more characteristics of the first column in the array of values; and
determine at least one of the set of embedding space parameters based on the one or more characteristics of the first column in the array of values.

3. The apparatus of claim 2 , wherein the one or more characteristics of the first column comprises a range of lengths for data values in the first column.

4. The apparatus of claim 1 , wherein the first column in the array of values is adjacent to the second column in the array of values.

5. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:
provide the set of object vectors as input to a machine learning algorithm; and
determine a classification of each data object in the set of data objects based on output of the machine learning algorithm in response to input of the set of object vectors.

6. The apparatus of claim 1 , wherein the instructions, when executed by the processor, further cause the processor to:
determine a third group of data objects in the set of data objects, the third group of data objects corresponding to a third column in the array of values that is adjacent to the first column or the second column in the array of values;
concatenate data values from the first group of data objects with the data values from one or more of the second group of data objects and the third group of data objects in a row-wise manner to produce a second concatenated group of data values; and
determine at least one of the set of embedding space parameters based on the second concatenated group of data values.

7. The apparatus of claim 6 , wherein the instructions, when executed by the processor, further cause the processor to:
determine a fourth group of data objects in the set of data objects, the fourth group of data objects corresponding to a fourth column in the array of values that is adjacent to the first column, the second column, or the third column in the array of values;
concatenate data values from the first group of data objects with the data values from the second group of data objects, the third group of data objects, and the fourth group of data objects in a row-wise manner to produce a third concatenated group of data values; and
determine at least one of the set of embedding space parameters based on the third concatenated group of data values.

8. The apparatus of claim 1 , wherein the plurality of dimensions that define the embedding space comprise two or more of a row dimension, a column dimension, a data value dimension, a column statistic dimension, and a concatenation dimension.

9. The apparatus of claim 1 , wherein the plurality of dimensions that define the embedding space comprise a plurality of concatenation dimensions and at least one column statistic dimension.

10. At least one non-transitory computer-readable medium comprising a set of instructions that, in response to execution by a processor circuit, cause the processor circuit to:
identify a set of data objects, the set of data objects comprising an array of values, wherein each data object in the set of data objects comprises a column value, a row value, and a data value;
determine a first group of data objects in the set of data objects, the first group of data objects corresponding to a first column in the array of values;
determine a second group of data objects in the set of data objects, the second group of data objects corresponding to a second column in the array of values that is adjacent to the first column in the array of values;
concatenate data values from the first group of data objects with the data values from the second group of data objects in a row-wise manner to produce a concatenated group of data values;
determine at least one of a set of a plurality of embedding space parameters based on the concatenated group of data values, wherein the set of embedding space parameters define an embedding space comprising a plurality of dimensions; and
generate a set of object vectors, the set of object vectors comprising an object vector for each data object in the set of data objects, each object vector in the set of object vectors to include a set of dimension values and each dimension value in the set of dimension values to correspond to one of the plurality of dimensions in the embedding space, wherein a respective object vector for a respective data object is generated based on a respective column value of the respective data object, a respective row value of the respective data object, and the set of embedding space parameters.

11. The at least one non-transitory computer-readable medium of claim 10 , wherein the set of instructions, in response to execution by the processor circuit, further cause the processor circuit to:
analyze each data value in the first group of data objects to determine one or more characteristics of the first column in the array of values; and
determine at least one of the set of embedding space parameters based on the one or more characteristics of the first column in the array of values.

12. The non-transitory computer-readable medium of claim 11 , wherein the one or more characteristics of the first column comprises a range of lengths for data values in the first column.

13. The non-transitory computer-readable medium of claim 10 , wherein the set of instructions, in response to execution by the processor circuit, further cause the processor circuit to:
determine a third group of data objects in the set of data objects, the third group of data objects corresponding to a third column in the array of values that is adjacent to the first column on an opposite side as the second column in the array of values;
concatenate data values from the first group of data object

Assignees

• Capital One Services Llc

Inventors

• Truong Anh
• Walters Austin Grant
• Goodsitt Jeremy Edward

Classifications

• G06N20/00 Primary

  Machine learning · CPC title