Systems and methods for facilitating data transformation

Patent metadata

Field	Value
Publication number	US-9946777-B1
Application number	US-201615383787-A
Country	US
Kind code	B1
Filing date	Dec 19, 2016
Priority date	Dec 19, 2016
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

Systems and methods are provided for facilitating the transformation of data from a tabular data set organized according to a data schema to an object based data set organized according to data ontology. The data schema and the data ontology may be used to generate a custom coding language suitable for facilitating the creation of a transform script. Error checking, type checking, autocomplete, and preview tools may be provided to permit the verification of the transform script as it is created.

First claim

The invention claimed is:

1. A system for performing an integration of an origin data set into a target data set, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
receive the origin data set, the origin data set being organized by an origin data set schema specifying a tabular format for data of the origin data set;
receive a target data set ontology, the target data set ontology defining data objects of the target data set;
generate, according to the origin data set schema and the target data set ontology, a domain-specific transform programming language specific to the origin data set schema and the target data set ontology;
receive transform instructions programmed in the domain-specific transform programming language;
generate a preview target data set from at least a portion of the origin data set using the transform instructions, the preview target data set generated in response to receiving the transform instructions; and
integrate the at least a portion of the origin data set into the target data set according to the received transform instructions in response to the preview target data set corresponding to the target data set ontology.

2. The system of claim 1 , wherein the system is further caused to:
store data transformation information about the integration, the data transformation information including information about the transform instructions and a time and date of integration.

3. The system of claim 2 , wherein to integrate the at least a portion of the origin data set into the target data set, the system is further caused to select the at least a portion of the origin data set according to the data transformation information and integrate the selected portion of the origin data set.

4. The system of claim 1 , wherein the preview target data set is generated while the system receives the transform instructions.

5. The system of claim 1 , wherein the system is further caused to provide a transform instructions template for receiving the transform instructions.

6. The system of claim 1 , wherein the system caused to receive transform instructions is further caused to:
analyze the received transform instructions for errors as they are received; and
notify a user of the errors.

7. The system of claim 6 , wherein to analyze the received transform instructions for errors the system is further caused to compare the received transform instructions to the target data set ontology and to the origin data set schema.

8. The system of claim 1 , wherein the system caused to receive the transform instructions is further caused to provide autocomplete suggestions.

9. A computer implemented method of integrating an origin data set into a target data set, the method being performed on a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, cause the computer system to perform the method, the method comprising:
receiving, by the computer system, the origin data set, the origin data set being organized by an origin data set schema specifying a tabular format for data of the origin data set;
receiving, by the computer system, a target data set ontology, the target data set ontology defining data objects of the target data set;
generating, by the computer system, according to the origin data set schema and the target data set ontology, a domain-specific transform programming language specific to the origin data set schema and the target data set ontology;
receiving, by the computer system, transform instructions programmed in the domain-specific transform programming language;
generating, by the computer system, a preview target data set from at least a portion of the origin data set using the transform instructions, the preview target data set generated in response to receiving the transform instructions; and
integrating, by the computer system, the at least a portion of the origin data set into the target data set according to the received transform instructions in response to the preview target data set corresponding to the target data set ontology.

10. The method of claim 9 , further comprising:
storing, by the computer system, data transformation information about the integration, the data transformation information including information about the transform instructions and a time and date of integration.

11. The method of claim 10 , wherein the integrating the at least a portion of the origin data set into the target data set includes selecting, by the computer system, the at least a portion of the origin data set according to the data transformation information and integrating the selected portion of the origin data set.

12. The method of claim 10 , wherein receiving the transform instructions includes
analyzing, by the computer system, the received transform instructions for errors as they are received; and
notifying, by the computer system, a user of the errors.

13. The method of claim 12 , wherein analyzing the received transform instructions for errors includes comparing the received transform instructions to the target data set ontology and to the origin data set schema.

14. The method of claim 9 , wherein the preview target data set is generated while the system receives transform instructions.

15. The method of claim 9 , further comprising providing, by the computer system, a transform instructions template for receiving the transform instructions.

16. The method of claim 9 , wherein receiving the transform instructions includes providing, by the computer system, autocomplete suggestions.

17. A system for performing an integration of an origin data set into a target data set, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
receive a target data set ontology, the target data set ontology defining data objects of the target data set;
access an origin data set schema, the origin data set schema including an organizational structure of the origin data set;
generate, according to the origin data set schema and the target data set ontology, a domain-specific transform programming language specific to the origin data set schema and the target data set ontology;
receive transform instructions from a user input device programmed in the domain-specific transform programming language; and
transfer the transform instructions to a remote computer system for integrating the origin data set into the target data set according to the received transform instructions.

18. The system of claim 17 , wherein the system is further caused to provide a preview target data set based on the received transform instructions.

Assignees

• Palantir Technologies Inc

Inventors

• Ossher Joel
• Xiao david

Classifications

• G06F16/21 Primary

  Design, administration or maintenance of databases · CPC title

• G06F16/25

  Integrating or interfacing systems involving database management systems · CPC title

• G06F16/212

  with details for data modelling support · CPC title

• G06F16/258 Primary

  Data format conversion from or to a database · CPC title

• G06F17/30294

  Physics · mapped topic