Automatically determining data dependencies to facilitate code execution

Patent metadata

Field	Value
Publication number	US-10671361-B2
Application number	US-201615333869-A
Country	US
Kind code	B2
Filing date	Oct 25, 2016
Priority date	Oct 25, 2016
Publication date	Jun 2, 2020
Grant date	Jun 2, 2020

Abstract

Technologies relating to determining data variable dependencies to facilitate code execution are disclosed. An example method includes: identifying a set of programming statements having a plurality of data parameters; identifying first data parameters associated with a first programming statement in the set of programming statements; determining one or more parameter dependencies associated with the first data parameters; and determining, based on the one or more parameter dependencies, a first execution performance of the first programming statement. The method may further include: determining a second execution performance of the second programming statement and scheduling execution of the first programming statement and of the second programming statement based on the first and second execution performances. For example, at least a portion of the first programming statement is executed in parallel to executing the second programming statement, or the first programming statement and the second programming statement are executed in a predefined sequence.

First claim

What is claimed is:

1. A system, comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving a programming code associated with an application for execution by the system, wherein the programming code comprises at least a first programming statement and a second programming statement;
determining that the first programming statement comprises a first dependency of a first variable and that the second programming statement comprises a second dependency of a second variable;
identifying, among a plurality of databases, a first database that stores first data corresponding to the first variable;
identifying, among the plurality of databases, a second database that stores second data corresponding to the second variable;
determining a first estimated time for accessing data from the first database by retrieving, from a stored table external to the application, execution statistics data associated with the first database based on the first variable;
determining that the second database is offline and thereby inaccessible by the programming code based on an attempt to execute the second programming statement;
in response to determining that the second database is inaccessible by the programming code, injecting, into the programming code, one or more programming statements for executing a batch data retrieval process;
determining a second estimated time for accessing the second data from the second database in the batch data retrieval process based at least on a data type and a data size associated with the second data and by executing a function that simulates an offline execution associated with retrieving the second data from the second database;
determining, based on the first estimated time and the second estimated time, an execution order for executing the programming code; and
executing a first instruction for retrieving the second data from the second database prior to executing a second instruction for retrieving the first data from the first database according to the determined execution order.

2. The system of claim 1 , wherein the determining that the first programming statement comprises the first dependency comprises parsing the first programming statement based on one or more parsing rules to identify the first variable.

3. The system of claim 2 , wherein the one or more parsing rules include at least one of a syntax-based parsing rule, a semantics-based parsing rule, a system-provided parsing rule, or a user defined parsing rule.

4. The system of claim 2 , wherein the one or more parsing rules include a parsing rule that is independent of a specific programming language.

5. The system of claim 1 , wherein the programming code further comprises a third programming statement, wherein the operations further comprise:
determining that the third programming statement comprises a third dependency of a third variable; and
determining, among the plurality of databases, that the second database stores third data corresponding to the third variable, wherein the one or more programming statements injected into the programming code specifies that the third data, along with the second data, is retrieved from the second database in the batch data retrieval process, and wherein the execution order further indicates that the second instruction and the third instruction are executed in parallel based on determining that both of the second programming statement and the third programming statement require data retrieval from the second database.

6. The system of claim 1 , wherein the determining the first estimated time comprises:
retrieving, from a stored table external to the application, execution statistics data associated with the first database based on the first variable.

7. The system of claim 1 , wherein the determining that the second programming statement comprises the second dependency comprises:
parsing the second programming statement; and
identifying a user-defined function based on the parsing, wherein the user-defined function comprises the second variable.

8. The system of claim 1 , wherein the determining the first second estimated time comprises executing a function to simulate an execution of the one or more programming statements associated with the offline batch data retrieval process to estimate a time required to retrieve data from the first second database.

9. The system of claim 7 , wherein the execution order indicates that the user-defined function included in the second programming statement is delayed for a predetermined amount of time based on the determining that the second database is inaccessible.

10. The system of claim 1 , wherein the programming code further comprises a third programming statement, wherein the operations further comprise:
determining that the third programming statement comprises a third dependency of a third variable;
determining, from the plurality of databases, that the second database that stores third data source corresponding to the third variable; and
modifying the one or more programming statements to include a programming statement for retrieving the third data from the second database in the offline batch data retrieval process.

11. A method, comprising:
obtaining, by one or more hardware processors, a programming code associated with an application for execution, wherein the programming code comprises at least a first programming statement and a second programming statement;
determining, by the one or more hardware processors, that the first programming statement comprises a first dependency of a first variable and that the second programming statement comprises a second dependency of a second variable;
identifying, by the one or more hardware processors among a plurality of databases, a first database that stores first data corresponding to the first variable;
generating a first instruction for retrieving the first data from the first database;
identifying, by the one or more hardware processors among the plurality of databases, a second database that stores second data corresponding to the second variable;
determining that the second database is offline and thereby inaccessible by the programming code based on an attempt to execute the second programming statement;
generating a second instruction for retrieving the second data from the second database in an offline batch data processing job;
in response to determining that the second database is offline and inaccessible injecting the first instruction and the second instruction into the programming code;
determining, by the one or more hardware processors, a first estimated time for accessing data from the first database by retrieving, from a stored table external to the application, execution statistics data associated with the first database based on the first variable;
determining a second estimated time for accessing the second data form the second databased in the offline batch data processing job based at least on a data type and a data size associated with the second data and by executing a function that simulates an offline execution associated with retrieving the second data from the second database;
determining, by the one or more hardware processors based on the first estimated time and the second estimated time, an execution sequence for executing the first and second instructions, wherein the execution sequence specifies that the second instruction is executed before the first instruction; and
executing, by the one or more hardware processors, the second instructio

Assignees

• Paypal Inc

Inventors

• Li Xin
• Deng Weijia
• Yang Shuan
• Chen Feng
• Yao Jin
• Ling Zhijun
• Li Yunfeng
• Yun Xiaohan
• Yu Yang

Classifications

• G06F8/427

  Parsing · CPC title

• G06F8/4441

  Reducing the execution time required by the program code · CPC title

• G06F8/45

  Exploiting coarse grain parallelism in compilation, i.e. parallelism between groups of instructions · CPC title

• G06F8/433 Primary

  Dependency analysis; Data or control flow analysis · CPC title