Tracking of code base and defect diagnostic coupling with automated triage

Patent metadata

Field	Value
Publication number	US-9824002-B2
Application number	US-201514755751-A
Country	US
Kind code	B2
Filing date	Jun 30, 2015
Priority date	Aug 16, 2011
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

In response to a test case error generated by execution of a test case against a code build, a source code segment that caused the test case error is identified by a defect monitor. The identified source code segment is linked to the test case that generated the test case error. The linked source code segment is monitored for code changes. A determination is made as to whether a test case re-execution criterion associated with the test case has been satisfied based upon a detected code change of the linked source code segment. An indication to re-execute the test case is generated in response to determining that the test case re-execution criterion associated with the test case has been satisfied.

First claim

What is claimed is:

1. A method, comprising:
by a defect monitor processor:
identifying, in response to detection of a test case error generated by execution of a test case against a code build, a source code function of the code build that caused the test case error;
evaluating, in response to a detected code change in the source code function, a test case re-execution criterion established in accordance with testing constraints within an automated testing environment to programmatically initiate re-execution of the test case, where the test case re-execution criterion comprises one of a test case priority of the test case and a length of execution time of the test case in combination with a code change threshold configured with the test case based upon the one of the test case priority of the test case and the length of execution time of the test case; and
initiating re-execution of the test case in response to determining that the test case re-execution criterion established in accordance with the testing constraints within the automated testing environment to programmatically initiate the re-execution of the test case has been satisfied.

2. The method of claim 1 , where identifying, in response to the detection of the test case error generated by execution of the test case against the code build, the source code function of the code build that caused the test case error comprises:
analyzing a stack trace associated with the test case error; and
determining, from the stack trace analysis, the source code function that caused the test case error.

3. The method of claim 1 , further comprising the defect monitor processor linking the source code function to the test case that generated the test case error by storing an identifier of the test case error, a source code link to the source code function, and a stack trace associated with the test case error as defect monitoring and tracking information.

4. The method of claim 1 , where determining that the test case re-execution criterion established in accordance with the testing constraints within the automated testing environment to programmatically initiate the re-execution of the test case has been satisfied comprises at least one of:
comparing a stack trace associated with the test case error with a new version of the source code function; and
one of:
determining whether the new version of the source code function has diverged from the identified source code function that caused the test case error by an amount established within the automated testing environment as sufficient to re-execute the test case; and
determining that the source code function as named in the stack trace has been one of deleted, renamed, and moved to a different source code file.

5. The method of claim 1 , further comprising the defect monitor processor:
receiving a result output from the re-execution of the test case; and
appending the result output to the test case.

6. The method of claim 1 , where the test case re-execution criterion comprises a proportional correlation between the test case priority of the test case and a defect severity associated with the test case error in further combination with an inverse correlation with the configured code change threshold to programmatically initiate the re-execution of the test case, where, as either of the test case priority and the defect severity are increased, the configured code change threshold to programmatically initiate the re-execution of the test case is decreased.

7. The method of claim 1 , where the test case re-execution criterion comprises a defect severity associated with the test case error and where the code change threshold configured with the test case comprises a number of changed lines of source code within the source code function selected according to the defect severity associated with the test case error.

8. A system, comprising:
a code repository; and
a hardware processor programmed to:
identify, in response to detection of a test case error generated by execution of a test case against a code build, a source code function of the code build within the code repository that caused the test case error;
evaluate, in response to a detected code change in the source code function within the code repository, a test case re-execution criterion established in accordance with testing constraints within an automated testing environment to programmatically initiate re-execution of the test case, where the test case re-execution criterion comprises one of a test case priority of the test case and a length of execution time of the test case in combination with a code change threshold configured with the test case based upon the one of the test case priority of the test case and the length of execution time of the test case; and
initiate re-execution of the test case in response to determining that the test case re-execution criterion established in accordance with the testing constraints within the automated testing environment to programmatically initiate the re-execution of the test case has been satisfied.

9. The system of claim 8 , where, in being programmed to identify, in response to the detection of the test case error generated by execution of the test case against the code build, the source code function of the code build that caused the test case error, the processor is programmed to:
analyze a stack trace associated with the test case error; and
determine, from the stack trace analysis, the source code function that caused the test case error.

10. The system of claim 8 , where the processor is further programmed to link the source code function to the test case that generated the test case error, comprising the processor being programmed to store an identifier of the test case error, a source code link to the source code function, and a stack trace associated with the test case error as defect monitoring and tracking information.

11. The system of claim 8 , where, in being programmed to determine that the test case re-execution criterion established in accordance with the testing constraints within the automated testing environment to programmatically initiate the re-execution of the test case has been satisfied, the processor is programmed to at least one of:
compare a stack trace associated with the test case error with a new version of the source code function; and
one of:
determine whether the new version of the source code function has diverged from the identified source code function that caused the test case error by an amount established within the automated testing environment as sufficient to re-execute the test case; and
determine that the source code function as named in the stack trace has been one of deleted, renamed, and moved to a different source code file.

12. The system of claim 8 , where the processor is further programmed to:
receive a result output from the re-execution of the test case; and
append the result output to the test case.

13. The system of claim 8 , where the test case re-execution criterion comprises a proportional correlation between the test case priority of the test case and a defect severity associated with the test case error in further combination with an inverse correlation with the configured code change threshold to programmatically initiate the re-execution of the test case, where, as either of the test case priority and the defect severity are increased, the configured code change threshold to programmatically initiate the re-execution of the test case is decreased.

14. The system of claim 8 , where the test case re-execution criterion comprises a defect severity as

Assignees

• IBM

Inventors

• Shann Katherine M
• Whitbourne Matthew D
• Would Daniel E
• Xu Shanna

Classifications

• G06F8/30

  Creation or generation of source code · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3612

  by runtime analysis (performance monitoring G06F11/3466) · CPC title

• G06F11/3664

  Physics · mapped topic

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title