Proactive and selective regression testing based on historic test results

Patent metadata

Field	Value
Publication number	US-10296446-B2
Application number	US-201514944271-A
Country	US
Kind code	B2
Filing date	Nov 18, 2015
Priority date	Nov 18, 2015
Publication date	May 21, 2019
Grant date	May 21, 2019

Abstract

In an approach to reducing regression test duration, one or more computer processors receive a first code change. The one or more computer processors retrieve one or more historic code changes relevant to the first code change. The one or more computer processors determine whether one or more failed test cases are relevant to the one or more historic code changes, where the one or more relevant failed test cases and the one or more historic code changes are stored in a table of test history. In response to determining the one or more historic code changes are relevant to one or more failed test cases, the one or more computer processors retrieve the one or more relevant failed test cases from the table of test history.

First claim

What is claimed is:

1. A computer program product for reducing regression test duration, the computer program product comprising:
one or more computer readable storage device and program instructions stored on the one or more computer readable storage device, the stored program instructions comprising:
program instructions to receive a first code change, wherein the first code change is a new code change to source code;
program instructions to determine that one or more historic code changes are relevant to the first code change based, at least in part, on a positional overlap at a line level;
program instructions to retrieve the one or more relevant historic code changes;
program instructions to determine whether the one or more relevant historic code changes caused one or more test cases to fail in the past, wherein the one or more test cases that failed in the past and the one or more relevant historic code changes are stored in a table of test history;
responsive to determining that the one or more relevant historic code changes caused the one or more test cases to fail in the past, program instructions to retrieve the one or more test cases that failed in the past from the table of test history;
program instructions to test the first code change with the retrieved one or more test cases that failed in the past;
program instructions to receive regression test results, wherein the regression test results include at least one failing test case;
program instructions to query the table of test history for the at least one failing test case; and
program instructions to determine whether at least one historic code change corresponding to the at least one failing test case is relevant to at least one current code change.

2. The computer program product of claim 1 , wherein the table of test history links one or more test cases that failed in the past to one or more historic code changes.

3. The computer program product of claim 2 , wherein the table of test history links corresponding metadata to the one or more test cases that failed in the past and the one or more relevant historic code changes, wherein the corresponding metadata includes at least one of: a date of occurrence, an associated defect, a severity of a failure, an identification of a coder that made a change, an age of an entry, a number of failing test cases attributed to a code change, a size of a code change, a scope of a code change, a level of experience of a submitting coder, and a time of day a change was submitted.

4. The computer program product of claim 1 , wherein the determined positional overlap is at one of a file level, an object level, or a component level.

5. The computer program product of claim 1 , wherein the program instructions to determine that at least one historic code change corresponding to the at least one failing test case is relevant to at least one current code change comprise program instructions to determine positional overlap of the at least one current code change to the at least one historic code change, wherein relevance is based on the determined positional overlap and is at one of a file level, a line level, an object level, or a component level.

6. The computer program product of claim 1 , wherein the table of test history explicitly links the one or more test cases that failed in the past to the one or more relevant historic code changes as rows in the table.

7. The computer program product of claim 1 , the stored program instructions further comprising, program instructions to rank the one or more test cases that failed in the past based, at least in part, on a level of relevance of the one or more relevant historic code changes.

8. A computer system for reducing regression test duration, the computer system comprising:
one or more computer processors;
one or more computer readable storage device;
program instructions stored on the one or more computer readable storage device for execution by at least one of the one or more computer processors, the stored program instructions comprising:
program instructions to receive a first code change, wherein the first code change is a new code change to source code;
program instructions to determine that one or more historic code changes are relevant to the first code change based, at least in part, on a positional overlap at a line level;
program instructions to retrieve the one or more relevant historic code changes;
program instructions to determine whether the one or more relevant historic code changes caused one or more test cases to fail in the past, wherein the one or more test cases that failed in the past and the one or more relevant historic code changes are stored in a table of test history;
responsive to determining that the one or more relevant historic code changes caused one or more test cases to fail in the past, program instructions to retrieve the one or more test cases that failed in the past from the table of test history;
program instructions to test the first code change with the retrieved one or more test cases that failed in the past;
program instructions to receive regression test results, wherein the regression test results include at least one failing test case;
program instructions to query the table of test history for the at least one failing test case; and
program instructions to determine whether at least one historic code change corresponding to the at least one failing test case is relevant to at least one current code change.

9. The computer system of claim 8 , wherein the table of test history links one or more test cases that failed in the past to one or more historic code changes.

10. The computer system of claim 9 , wherein the table of test history links corresponding metadata to the one or more test cases that failed in the past and the one or more relevant historic code changes, wherein the corresponding metadata includes at least one of: a date of occurrence, an associated defect, a severity of a failure, an identification of a coder that made a change, an age of an entry, a number of failing test cases attributed to a code change, a size of a code change, a scope of a code change, a level of experience of a submitting coder, and a time of day a change was submitted.

11. The computer system of claim 8 , wherein the determined positional overlap is at one of a file level, an object level, or a component level.

12. The computer system of claim 8 , wherein the program instructions to determine that at least one historic code change corresponding to the at least one failing test case is relevant to at least one current code change comprise program instructions to determine positional overlap of the at least one current code change to the at least one historic code change, wherein relevance is based on the determined positional overlap and is at one of a file level, a line level, an object level, or a component level.

13. The computer system of claim 8 , wherein the table of test history explicitly links the one or more test cases that failed in the past to the one or more relevant historic code changes as rows in the table.

14. The computer system of claim 8 , the stored program instructions further comprising, program instructions to rank the one or more test cases that failed in the past based, at least in part, on a level of relevance of the one or more relevant historic code changes.

Assignees

• IBM

Inventors

• D'Andrea Adam K
• Elliott Jenna R
• Khan Fahad A
• Kwong Raymond C
• Mcintyre Chad J
• Sadeghian Sadeghabad Sina

Classifications

• G06F11/3672

  Test management · CPC title

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3692

  for test results analysis · CPC title