Generation of optimized logic from a schema

Patent metadata

Field	Value
Publication number	US-11734264-B2
Application number	US-202117558097-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Mar 6, 2020
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

A method includes accessing a schema that specifies relationships among datasets, computations on the datasets, or transformations of the datasets, selecting a dataset from among the datasets, and identifying, from the schema, other datasets that are related to the selected dataset. Attributes of the datasets are identified, and logical data representing the identified attributes and relationships among the attributes is generated. The logical data is provided to a development environment, which provides access to portions of the logical data representing the identified attributes. A specification that specifies at least one of the identified attributes in performing an operation is received from the development environment. Based on the specification and the relationships among the identified attributes represented by the logical data, a computer program is generated to perform the operation by accessing, from storage, at least one dataset having the at least one of the attributes specified in the specification.

First claim

What is claimed is:

1. A method implemented by a data processing system for providing one or more interfaces and storage that stores datasets having one or more attributes, and with the one or more interfaces providing access to the one or more attributes of the datasets, including:
accessing a schema that specifies relationships among datasets represented in the schema, one or more computations on one or more of the datasets, or one or more transformations of one or more of the datasets;
identifying, from among the datasets, a plurality of the datasets in storage, by:
selecting a dataset from among the datasets; and

identifying, from the schema, one or more other datasets that are related to the selected dataset;
identifying attributes of the plurality of the datasets;
generating logical data representing identified attributes of the plurality of the datasets and further representing one or more relationships among the attributes;
providing, through one or more interfaces, access to one or more portions of the logical data representing the identified attributes of the plurality of the datasets;
receiving, from the one or more interfaces, a specification that specifies at least one of the identified attributes in performing an operation; and
based on the specification and on the one or more relationships among the identified attributes represented by the logical data, generating a computer program that is configured to perform the operation by accessing, from storage, at least one dataset from the plurality, with the at least one dataset accessed having the at least one of the attributes specified in the specification.

2. The method of claim 1 , wherein the one or more interfaces provides access to the one or more portions of the logical data without accessing the plurality of datasets from storage.

3. The method of claim 1 , comprising:
identifying a dataset from the plurality of datasets including the at least one of the attributes specified in the specification; and
accessing, from storage, the identified dataset.

4. The method of claim 1 , comprising executing the computer program using the at least one dataset accessed from storage.

5. The method of claim 1 , comprising optimizing the computer program to produce an optimized computer program that is configured to perform the operation by accessing, from storage, only those datasets in the plurality of datasets having the at least one of the attributes specified in the specification.

6. The method of claim 1 , wherein the one or more attributes include field names of the plurality of the datasets.

7. The method of claim 1 , wherein the one or more attributes include information for accessing the plurality of the datasets in storage.

8. The method of claim 1 , comprising identifying, from the schema, one or more parameters for joining the selected dataset and the one or more other datasets.

9. The method of claim 8 , wherein the one or more parameters include a key for joining the selected dataset and at least one of the one or more other datasets.

10. The method of claim 1 , comprising receiving, from a client device, selection data specifying the selected dataset.

11. The method of claim 1 , wherein the selected dataset comprises a root node of the logical data, and wherein at least one of the one or more other datasets are joined to the selected dataset.

12. The method of claim 1 , wherein the one or more computations on one or more of the datasets or one or more transformations of one or more of the datasets define a virtual field for at least one of the plurality of the datasets.

13. The method of claim 1 , comprising generating, based on the specification and on the one or more relationships among the identified attributes represented by the logical data, an executable dataflow graph that is configured to perform the operation, wherein the executable dataflow graph includes at least one of the one or more attributes as an input.

14. The method of claim 1 , comprising removing from the computer program an operation to access, from storage, at least one dataset in the plurality of datasets that does not include the at least one of the attributes specified in the specification.

15. The method of claim 1 , wherein the computer program is configured to access, from storage, at least some data from the plurality by a select statement, wherein the select statement is minimized to select only the at least one of the attributes specified in the specification.

16. The method of claim 1 , wherein the logical data is read as a data source.

17. The method of claim 1 , further including:
providing a development environment that includes one or more graphical user interfaces for viewing the one or more portions of the logical data representing the identified attributes of the plurality of the datasets and further for defining the specification that specifies at least one of the identified attributes in performing the operation.

18. A system for providing one or more interfaces and storage that stores datasets having one or more attributes, and with the one or more interfaces providing access to the one or more attributes of the datasets, including:
one or more processors and one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform operations comprising:
accessing a accessing a schema that specifies relationships among datasets represented in the schema, one or more computations on one or more of the datasets, or one or more transformations of one or more of the datasets;
identifying, from among the datasets, a plurality of the datasets in storage, by:
selecting a dataset from among the datasets; and
identifying, from the schema, one or more other datasets that are related to the selected dataset;

identifying attributes of the plurality of the datasets;
generating logical data representing identified attributes of the plurality of the datasets and further representing one or more relationships among the attributes;
providing, through one or more interfaces, access to one or more portions of the logical data representing the identified attributes of the plurality of the datasets;
receiving, from the one or more interfaces, a specification that specifies at least one of the identified attributes in performing an operation; and
based on the specification and on the one or more relationships among the identified attributes represented by the logical data, generating a computer program that is configured to perform the operation by accessing, from storage, at least one dataset from the plurality, with the at least one dataset accessed having the at least one of the attributes specified in the specification.

19. The system of claim 18 , wherein the computer program is configured to access, from storage, only those datasets having the at least one of the attributes specified in the specification.

20. A non-transitory computer-readable storage medium storing instructions for causing a computing system to:
access a schema that specifies relationships among datasets represented in the schema, one or more computations on one or more of the datasets, or one or more transformations of one or more of the datasets;
identify, from among the datasets, a plurality of the datasets in storage, by:
selecting a dataset from among the datasets; and
identifying, from the schema, one or more other datasets that are related to the selected dataset;

identify attribute

Assignees

• Ab Initio Technology Llc

Inventors

• Egenolf Jonah
• Isman Marshall A
• Schechter Ian

Classifications

• G06F16/81

  Indexing, e.g. XML tags; Data structures therefor; Storage structures · CPC title

• G06F16/9027

  Trees · CPC title

• G06F16/2428

  Query predicate definition using graphical user interfaces, including menus and forms (G06F16/2423 takes precedence) · CPC title

• G06F8/34

  Graphical or visual programming · CPC title

• G06F16/25 Primary

  Integrating or interfacing systems involving database management systems · CPC title