System and method for performing parallel and distributed analysis of program code to generate dependency graphs for executing extract transform load transformations

Patent metadata

Field	Value
Publication number	US-11842176-B2
Application number	US-202117525035-A
Country	US
Kind code	B2
Filing date	Nov 12, 2021
Priority date	Nov 12, 2021
Publication date	Dec 12, 2023
Grant date	Dec 12, 2023

Abstract

Embodiments of the present invention provide a system for performing parallel and distributed analysis of program code to generate directed graphs for executing extract transform load transformations. The system is configured for identifying that a user has initiated a request for analysis of a program code, via a compiler, extracting a syntax tree associated with the program code from the compiler, performing parallel and distributed analysis of the program code based on the syntax tree, generating a directed graph based on performing parallel and distributed analysis of the program code, and storing the directed graph in a cache memory.

First claim

The invention claimed is:

1. A system for performing parallel and distributed analysis of program code to generate directed graphs for executing extract transform load transformations, the system comprising:
at least one network communication interface;
at least one non-transitory storage device; and
at least one processing device coupled to the at least one non-transitory storage device and the at least one network communication interface, wherein the at least one processing device is configured to:
identify that a user has initiated request for analysis of a program code, via a compiler interface associated with a compiler;
extract a syntax tree associated with the program code from the compiler;
perform parallel and distributed analysis of the program code based on the syntax tree;
generate a directed graph based on performing parallel and distributed analysis of the program code, wherein the directed graph comprises metadata associated with the program code and calculations associated with each of one or more nodes associated with the directed graph;
store the directed graph in a cache memory,
identify that the user has initiated a second request for analysis of the program code, via the compiler interface;
determine, via a smart detection module, if cached directed graph stored in the cache memory is valid; and
allow or deny usage of the cached directed graph for performing transformation of data associated with the program code based on determining if the cached directed graph is valid.

2. The system of claim 1 , wherein the at least one processing device is configured to:
determine, via the smart detection module, that cached directed graph stored in the cache memory is valid; and
load the cached directed graph stored in the cache memory to perform the transformation of data associated with the program code.

3. The system of claim 1 , wherein the at least one processing device is configured to:
determine, via the smart detection module, that cached directed graph stored in the cache memory is not valid;
extract the syntax tree associated with the program code from the compiler;
perform parallel and distributed analysis of the program code based on the syntax tree;
update the directed graph based on performing parallel and distributed analysis of the program code; and
use the updated directed graph to perform the transformation of data associated with the program code.

4. The system of claim 1 , wherein the at least one processing device is configured to perform the parallel and distributed analysis of the program code based on:
scanning the program code to identify classes associated with the calculations used in the transformation of data associated with the program code; and
scanning the program code associated with the classes in parallel to identify the calculations, one or more dependencies, and one or more cyclic dependencies, wherein the generated directed graph is based on the calculations, the one or more dependencies, and the one or more cyclic dependencies.

5. The system of claim 4 , wherein the at least one processing device is configured to perform the transformation of data associated with the program code in parallel based on the identified calculations, the one or more dependencies, and the one or more cyclic dependencies.

6. The system of claim 5 , wherein the at least one processing device is configured to display results associated with the transformation of data to one or more users.

7. The system of claim 4 , wherein the at least one processing device is configured to optimize execution of one or more transformation operations associated with the transformation of data based on the one or more dependencies and the one or more cyclic dependencies.

8. A computer program product for performing parallel and distributed analysis of program code to generate directed graphs for executing extract transform load transformations, the computer program product comprising a non-transitory computer-readable storage medium having computer executable instructions for causing a computer processor to perform the steps of:
identifying that a user has initiated a request for analysis of a program code, via a compiler interface associated with a compiler;
extracting a syntax tree associated with the program code from the compiler;
performing parallel and distributed analysis of the program code based on the syntax tree;
generating a directed graph based on performing parallel and distributed analysis of the program code, wherein the directed graph comprises metadata associated with the program code and calculations associated with each of one or more nodes associated with the directed graph;
storing the directed graph in a cache memory;
identifying that the user has initiated a second request for analysis of the program code, via the compiler interface;
determining, via a smart detection module, if cached directed graph stored in the cache memory is valid; and
allowing or denying usage of the cached directed graph for performing transformation of data associated with the program code based on determining if the cached directed graph is valid.

9. The computer program product of claim 8 , wherein the computer executable instructions cause the computer processor to perform the steps of:
determining, via the smart detection module, that cached directed graph stored in the cache memory is valid; and
loading the cached directed graph stored in the cache memory to perform the transformation of data associated with the program code.

10. The computer program product of claim 8 , wherein the computer executable instructions cause the computer processor to perform the steps of:
determining, via the smart detection module, that cached directed graph stored in the cache memory is not valid;
extract the syntax tree associated with the program code from the compiler;
performing parallel and distributed analysis of the program code based on the syntax tree;
updating the directed graph based on performing parallel and distributed analysis of the program code; and
using the updated directed graph to perform the transformation of data associated with the program code.

11. The computer program product of claim 8 , wherein the computer executable instructions cause the computer processor to perform the step of performing the parallel and distributed analysis of the program code based on:
scanning the program code to identify classes associated with the calculations used in the transformation of data associated with the program code; and
scanning the program code associated with the classes in parallel to identify the calculations, one or more dependencies, and one or more cyclic dependencies, wherein the generated directed graph is based on the calculations, the one or more dependencies, and the one or more cyclic dependencies.

12. The computer program product of claim 11 , wherein the computer executable instructions cause the computer processor to perform the step of performing the transformation of data associated with the program code in parallel based on the identified calculations, the one or more dependencies, and the one or more cyclic dependencies.

13. The computer program product of claim 12 , wherein the computer executable instructions cause the computer processor to perform the step of displaying results associated with the transformation of data to one or more users.

14. The computer program product of claim 11 , wherein the computer executable instructions cause the computer processor to perform the step of optimizing execution of one or more transformation operations associated with th

Assignees

• Bank Of America

Inventors

• Ahmed Malik Faisal
• Daniel Manoj S
• Dev Sachin
• Karra Rakesh Reddy
• Kondabolu Kushvanth
• Sesha Srinivasa Varada Pawan Kumar Kodakandla

Classifications

• G06F8/433 Primary

  Dependency analysis; Data or control flow analysis · CPC title

• G06F8/427

  Parsing · CPC title

• G06F8/451

  Code distribution (considering CPU load at run-time G06F9/505; load rebalancing G06F9/5083) · CPC title