Managing parameter sets

Patent metadata

Field	Value
Publication number	US-10318283-B2
Application number	US-201514803374-A
Country	US
Kind code	B2
Filing date	Jul 20, 2015
Priority date	Jul 18, 2014
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

Managing sets of parameter values includes: receiving a plurality of sets of parameter values for a generic computer program, and processing log entries associated with executions of instances of the generic computer program, each instance associated with one or more parameter values. The processing includes: analyzing the generic computer program to classify each of one or more parameters associated with the generic computer program as a member of either a first class or a second class; processing a log entry associated with an execution of a first instance of the generic computer program to form a particular set of parameter values; and determining whether to add the particular set of parameter values to the plurality of sets of parameter values based on a comparison of a first identifier for the particular set of parameter values to identifiers for at least some of the sets of parameter values.

First claim

What is claimed is:

1. A method for managing sets of parameter values, the method including:
receiving a plurality of sets of parameter values for a generic computer program, and
processing log entries associated with executions of instances of the generic computer program, each instance of the generic computer program being associated with one or more parameter values the processing including:
analyzing the generic computer program to classify each of one or more parameters associated with the generic computer program as a member of either a first class of parameters or a second class of parameters, including classifying parameters based on whether or not the parameters affect data lineage associated with the generic computer program, the data lineage representing dependency relationships arising from execution of one or more instances of the generic computer program;
processing a log entry associated with an execution of a first instance of the generic computer program to form a particular set of parameter values, wherein the processing includes:
including, in the particular set, any values of parameters classified as members of the first class occurring in the log entry, and
excluding, from the particular set, any values of parameters classified as members of the second class occurring in the log entry; and

determining whether to add the particular set of parameter values to the plurality of sets of parameter values based on a comparison of a first identifier for the particular set of parameter values to identifiers for at least some of the sets of parameter values of the plurality of sets of parameter values.

2. The method of claim 1 wherein the comparison of the first identifier for the particular set of parameter values to the identifiers for at least some of the sets of parameter values of the plurality of sets of parameter values includes:
determining the first identifier based on the particular set of parameter values and an identifier of the generic computer program;
determining a plurality of second identifiers, one for each set of parameter values of the at least some of the sets of parameter values; and
comparing the first identifier to each second identifier of the plurality of second identifiers to determine whether the first identifier and any of the second identifiers match.

3. The method of claim 2 wherein determining whether to add the particular set of parameter values to the plurality of sets of parameter values includes determining to add the particular set of parameter values to the plurality of sets of parameter values if none of the second identifiers match the first identifier.

4. The method of claim 2 wherein determining the first identifier includes computing an identification string from the contents of the particular set of parameter values, and determining the second identifiers includes computing identification strings from the contents of the at least some of the sets of parameter values of the plurality of sets of parameter values.

5. The method of claim 2 wherein determining the first identifier includes forming a concatenation of one or more of the identifier of the generic computer program, name-value pairs of the particular set of parameter values, a function prototype of the generic computer program, and a project scope for the first instance of the generic computer program.

6. The method of claim 2 wherein determining the first identifier includes applying a data mapping function to one or more of the identifier of the generic computer program, name-value pairs of the particular set of parameter values, a function prototype of the generic computer program, and a project scope for the first instance of the generic computer program.

7. The method of claim 6 wherein the data mapping function includes a hash function.

8. The method of claim 1 wherein the first class of parameters includes parameters that affect the logical operation of the generic computer program and the second class of parameters includes parameters that do not affect the logical operation of the generic computer program.

9. The method of claim 1 wherein the generic computer program is specified as a dataflow graph that includes nodes representing data processing operations and links between the nodes representing flows of data elements between the data processing operations.

10. The method of claim 1 wherein, for each of the parameters, the analysis includes automatically classifying the parameter.

11. The method of claim 10 wherein automatically classifying the parameter includes initially classifying the parameter as belonging to the first class of parameters, determining a number of unique values of the parameter in a plurality of executions of instances of the generic computer program, and re-classifying the parameter as belonging to the second class of parameters if the number of unique values of the parameter exceeds a predetermined threshold.

12. The method of claim 10 wherein automatically classifying the parameter includes initially classifying the parameter as belonging to the first class of parameters, determining whether or not changes in values of the parameter in a plurality of executions of instances of the generic computer program affect data lineage associated with the generic computer program, and re-classifying the parameter as belonging to the second class of parameters if changes in values of the parameter do not affect the data lineage.

13. The method of claim 1 further including forming an association between the log entry associated with the execution of the first instance of the generic computer program and the particular set of parameter values.

14. The method of claim 1 wherein the log entry associated with the execution of the first instance of the generic computer program includes a log entry of an execution command used to instantiate the generic computer program including one or more parameter values supplied as arguments to the execution command.

15. The method of claim 14 wherein the log entry associated with the execution of the first instance of the generic computer program further includes one or more of an indication of the project that the first instance executed in, an indication of internal parameters for the first instance, and an indication of environmental settings, global variables, and configuration variables used by the first instance.

16. The method of claim 10 further including processing an overall plurality of sets of parameter values for a plurality of generic computer programs and an overall plurality of log entries associated with executions of instances of at least some of the plurality of generic computer programs to form a data lineage report, wherein the overall plurality of sets of parameter values includes the augmented plurality of sets of parameter values for the generic computer program, and the overall plurality of log entries associated with executions of instances of the at least some of the plurality of generic computer programs includes the log entry of the execution of the first instance of the generic computer program, including its association with the particular set of parameter values.

17. The method of claim 16 , wherein forming the data lineage report includes, for each set of parameter values of the overall plurality of sets of parameter values for the plurality of generic computer programs,
processing the overall plurality of log entries associated with executions of instances of the at least some of the plurality of generic computer programs to identify all log entries associated with

Assignees

• Ab Initio Technology Llc

Inventors

• Bach Edward
• Oberdorf Richard
• Larson Brond

Classifications

• G06Q10/0633

  Workflow analysis · CPC title

• G06F9/44505

  Configuring for program initiating, e.g. using registry, configuration files · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F9/4494

  data driven · CPC title

• G06F8/70 Primary

  Software maintenance or management · CPC title