Extraction of problem diagnostic knowledge from test cases

Patent metadata

Field	Value
Publication number	US-11169906-B2
Application number	US-201916267709-A
Country	US
Kind code	B2
Filing date	Feb 5, 2019
Priority date	Jul 10, 2014
Publication date	Nov 9, 2021
Grant date	Nov 9, 2021

Abstract

Embodiments of the present invention enable users to extract knowledge from testing scenarios performed during application development, and later employ that knowledge to interpret application usage scenarios to enhance serviceability of applications by expediting identification and solving of problems. In an exemplary embodiment, log data generated during simulation of test cases is analyzed to create one or more rules based on patterns in which one or more log entries appear in the log data. Later, log data may be analyzed to look for a pattern of log entries that matches a pattern-based rule, thereby facilitating faster identification and resolution of the problem.

First claim

What is claimed is:

1. A method, the method comprising:
identifying, by one or more processors, a first set of log entries within a first set of log data corresponding to a first execution of a test case of a software application;
creating, by the one or more processors, a first rule based, at least in part, on a chronological order and content of log entries in the first set of log entries, where the first rule is associated with stored knowledge about the test case including: (i) information pertaining to the test case and (ii) application code from which each log entry of the first set of log entries resulted; and
responsive to identification of at least a partial match between the first rule and a second pattern of program activity, generating, by the one or more processors, a message that includes stored knowledge about the test case.

2. The method of claim 1 , wherein the content of log entries exclusively includes log message identifiers based, at least in part, on user input specifying which content of the first set of log entries is usable for creating the first rule.

3. The method of claim 1 ,
wherein the application code from which each log entry of the first set of log entries resulted corresponds to specific lines of code of the software application.

4. The method of claim 1 , further comprising:
receiving the first set of log data from a first development environment executing the first test case of the software application.

5. The method of claim 1 ,
wherein the second pattern of program activity corresponds to a second set of log entries within a second set of log data corresponding to a second execution of the software application.

6. The method of claim 1 , wherein the second execution of the software application corresponds to usage of the software application by an end-user.

7. The method of claim 1 ,
wherein the information pertaining to the test case includes information indicative of: (i) an identification number of the test case, (ii) descriptive information of which aspects of the software application were tested by the test case, (iii) expected results of the test case, and (iv) known issues of the software application.

8. A computer program product (CPP) comprising:
a computer readable storage device; and
computer code stored on the computer readable storage device, with the computer code including instructions for causing a processor(s) set to perform operations including the following:
identifying a first set of log entries within a first set of log data corresponding to a first execution of a test case of a software application,
creating a first rule based, at least in part, on a chronological order and content of log entries in the first set of log entries, where the first rule is associated with stored knowledge about the test case including: (i) information pertaining to the test case and (ii) application code from which each log entry of the first set of log entries resulted, and
responsive to identification of at least a partial match between the first rule and a second pattern of program activity, generating a message that includes stored knowledge about the test case.

9. The CPP of claim 8 , wherein the content of log entries exclusively includes log message identifiers based, at least in part, on user input specifying which content of the first set of log entries is usable for creating the first rule.

10. The CPP of claim 8 , wherein the application code from which each log entry of the first set of log entries resulted corresponds to specific lines of code of the software application.

11. The CPP of claim 8 , wherein the computer code further includes instructions for causing the processor(s) set to perform the following operations:
receiving the first set of log data from a first development environment executing the first test case of the software application.

12. The CPP of claim 8 , wherein the second pattern of program activity corresponds to a second set of log entries within a second set of log data corresponding to a second execution of the software application.

13. The CPP of claim 8 , wherein the information pertaining to the test case includes information indicative of: (i) an identification number of the test case, (ii) descriptive information of which aspects of the software application were tested by the test case, (iii) expected results of the test case, and (iv) known issues of the software application.

14. A computer system (CS) comprising:
a processor(s) set;
a computer readable storage device; and
computer code stored on the computer readable storage device, with the computer code including instructions for causing the processor(s) set to perform operations including the following:
identifying a first set of log entries within a first set of log data corresponding to a first execution of a test case of a software application,
creating a first rule based, at least in part, on a chronological order and content of log entries in the first set of log entries, where the first rule is associated with stored knowledge about the test case including: (i) information pertaining to the test case and (ii) application code from which each log entry of the first set of log entries resulted, and
responsive to identification of at least a partial match between the first rule and a second pattern of program activity, generating a message that includes stored knowledge about the test case.

15. The CS of claim 14 , wherein the content of log entries exclusively includes log message identifiers based, at least in part, on user input specifying which content of the first set of log entries is usable for creating the first rule.

16. The CS of claim 14 , wherein the application code from which each log entry of the first set of log entries resulted corresponds to specific lines of code of the software application.

17. The CS of claim 14 , wherein the computer code further includes instructions for causing the processor(s) set to perform the following operations:
receiving the first set of log data from a first development environment executing the first test case of the software application.

18. The CS of claim 14 , wherein the second pattern of program activity corresponds to a second set of log entries within a second set of log data corresponding to a second execution of the software application.

19. The CS of claim 14 , wherein the information pertaining to the test case includes information indicative of: (i) an identification number of the test case, (ii) descriptive information of which aspects of the software application were tested by the test case, (iii) expected results of the test case, and (iv) known issues of the software application.

Assignees

• IBM

Inventors

• Krishna Hariharan
• Ramakrishnan Arun
• Shetty Rohit

Classifications

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06N5/025

  Extracting rules from data · CPC title

• G06F11/3692

  for test results analysis · CPC title