Data logging for rule specifications

Patent metadata

Field	Value
Publication number	US-9547638-B2
Application number	US-49531609-A
Country	US
Kind code	B2
Filing date	Jun 30, 2009
Priority date	Jun 30, 2008
Publication date	Jan 17, 2017
Grant date	Jan 17, 2017

Abstract

At least one rule specification is received for a graph-based computation having data processing components connected by linking elements representing data flows. The rule specification defines rules that are each associated with one or more rule cases that specify criteria for determining one or more output values that depend on input data. A transform is generated for at least one data processing component in the graph-based computation based on the received rule specification, including providing an interface for configuring characteristics of a log associated with the generated transform. At least one data flow is transformed using the generated transform, including: tracing execution of the data processing components in the graph-based computation at run time, generating log information based on the traced execution according to the configured log characteristics, and storing or outputting the generated log information.

First claim

What is claimed is:

1. A method, including:
receiving at least one rule specification, that specifies criteria for determining one or more output values that depend on input data;
providing an interface for identifying conditions for generating log messages;
generating output records by transforming input data according to the rule, including:
determining that at least one of the conditions has occurred,
tracing the transforming, and
in response to determining that at least one of the conditions has occurred, generating log messages based on the tracing, the log messages including information not included in the output records;

providing the output records on a first channel; and
providing the log messages on a second channel different from the first channel.

2. The method of claim 1 in which at least one of the conditions is associated with transforming records in the input data according to a rule associated with the received rule specification.

3. The method of claim 2 in which the at least one of the conditions includes identifying an error in a record to be transformed.

4. The method of claim 2 in which the at least one of the conditions includes satisfying a condition of a rule case for a given record.

5. The method of claim 4 in which satisfying a condition of the rule case for a given record includes satisfying a logical expression based on values in the given record.

6. The method of claim 4 in which satisfying a condition of a rule case for a given record includes comparing values in the record to values associated with the rule case.

7. The method of claim 1 in which generating log messages includes:
generating one or more legend log messages each including details of a rule set containing the rules defined by the received rule specification, and
generating multiple tracing log messages, where each tracing log message is associated with a legend record and describes at least one event associated with transforming records in the input data according to a rule associated with the received rule specification.

8. The method of claim 7 in which a tracing log message that describes the event describes at least one input or output of a data processing component using an index in the legend record.

9. The method of claim 7 in which generating one or more legend messages includes generating one legend message per execution of a graph-based computation.

10. The method of claim 1 further including generating log messages for each rule for which the specified criteria are satisfied.

11. The method of claim 1 in which providing the log messages on a second channel includes outputting log messages from a log port of one or more data processing components.

12. The method of claim 11 in which providing the log messages on a second channel further includes receiving log messages from the log port in a data processing component and storing log information derived at least in part from the log messages.

13. The method of claim 12 further including filtering the received log messages and storing log information derived from a subset of the log messages.

14. The method of claim 11 further including:
receiving log messages from the log port in a data processing component that indicate rule for which the specified criteria are satisfied,
examining the log messages to determine a reduced set of input records that provide at least one log message for each rule case of each of the rules defined by the rule specification that is triggered by all of the input records, and
storing the reduced set of input records.

15. A computer system, including:
a storage system storing at least one rule specification, that specifies criteria for determining one or more output values that depend on input data;
an interface for identifying conditions for generating log messages; and
a computation system configured to:
generate output records by transforming input data according to the rule, including:
determining that at least one of the conditions has occurred,
tracing the transforming, and
generating, in response to determining that at least one of the conditions has occurred, log messages based on the tracing, the log messages including information not included in the output records;

provide the output records on a first channel; and
provide the log messages on a second channel different from the first channel.

16. The computer system of claim 15 in which at least one of the conditions is associated with transforming records in the input data according to a rule associated with the received rule specification.

17. The computer system of claim 16 in which the at least one of the conditions includes identifying an error in a record to be transformed.

18. The computer system of claim 16 in which the at least one of the conditions includes satisfying a condition of a rule case for a given record.

19. The computer system of claim 18 in which satisfying a condition of the rule case for a given record includes satisfying a logical expression based on values in the given record.

20. The computer system of claim 18 in which satisfying a condition of a rule case for a given record includes comparing values in the record to values associated with the rule case.

21. The computer system of claim 15 in which generating log messages includes:
generating one or more legend log messages each including details of a rule set containing the rules defined by the received rule specification, and
generating multiple tracing log messages, where each tracing log message is associated with a legend record and describes at least one event associated with transforming records in the input data according to a rule associated with the received rule specification.

22. The computer system of claim 21 in which a tracing log message that describes the event describes at least one input or output of a data processing component using an index in the legend record.

23. The computer system of claim 21 in which generating one or more legend messages includes generating one legend message per execution of a graph-based computation.

24. The computer system of claim 15 in which the computer system is further configured to generate log messages for each rule case for which the specified criteria are satisfied.

25. The computer system of claim 15 in which providing the log messages on a second channel includes outputting log messages from a log port of one or more data processing components.

26. The computer system of claim 25 in which providing the log messages on a second channel further includes receiving log messages from the log port in a data processing component and storing log information derived at least in part from the log messages.

27. The computer system of claim 26 in which the computer system is further configured to filter the received log messages and store log information derived from a subset of the log messages.

28. The computer system of claim 25 in which the computer system is further configured to:
receive log messages from the log port in a data processing component that indicate rule for which the specified criteria are satisfied,
examine the log messages to determine a reduced set of input records that provide at least one log message for each rule case of each of the rules defined by the rule specification that is triggered by all of the input

Assignees

• Studer Scott
• Gould Joel
• Phillimore David
• Ab Initio Technology Llc

Inventors

• Studer Scott
• Gould Joel
• Phillimore David

Classifications

• G06F17/40

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F40/18 Primary

  of spreadsheets (form-filling G06F40/174) · CPC title

• G06F8/00 Primary

  Arrangements for software engineering (testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06) · CPC title

• G06F21/552

  involving long-term monitoring or reporting · CPC title

• G06Q10/10

  Office automation; Time management · CPC title