Focus area integration test heuristics

Patent metadata

Field	Value
Publication number	US-10394697-B2
Application number	US-201715595571-A
Country	US
Kind code	B2
Filing date	May 15, 2017
Priority date	May 15, 2017
Publication date	Aug 27, 2019
Grant date	Aug 27, 2019

Abstract

Techniques are disclosed for identifying an optimal set of test cases of a larger test case bucket to execute on a new software release. The release includes one or more patches modifying development code of a previous release of the software, and each test case includes metadata specifying a history of the test case relative to an execution on the previous release. In each patch, it is determined, based on the metadata, whether at a test case targets the development code modified by the patch. Upon determining that the test case targets the development code, a score associated with the test case is incremented. The test cases are prioritized for execution based on score.

First claim

What is claimed is:

1. A computer-implemented method to reduce a processing overhead incurred in testing areas of development code impacted by a target release of software, the computer-implemented method comprising:
receiving a test case bucket containing a plurality of test cases for executing on the target release of the software, wherein the target release includes one or more patches, each patch of which modifies a respective area of development code of a previous release of the software, wherein each test case is associated with a respective, candidate score and specifies a history of the respective test case relative to an execution on the previous release;
upon determining, for each of the one or more patches, a respective test case, of the plurality of test cases, that targets the area of development code modified by the respective patch, incrementing the candidate score associated with the respective test case;
for each patch, identifying one or more developers associated with the respective patch based on metadata of the respective patch;
evaluating a test case history associated with the respective one or more developers to determine existence of one or more test cases that have previously failed and that are associated with the respective one or more developers;
upon determining the existence of the one or more test cases, obtaining identifying information for the one or more test cases based on the metadata and incrementing a score for each of the one or more test cases, wherein the one or more test cases are included in the test case bucket; and
prioritizing, by operation of one or more computer processors and based on the score of each test case, execution of the test cases in the test case bucket during execution of the test case bucket on the target release of the software, wherein the test case having the highest score from the test case bucket is selected and executed first.

2. The computer-implemented method of claim 1 , wherein each test case further specifies a respective code coverage measure and a respective impact measure of the test case.

3. The computer-implemented method of claim 1 , wherein the computer-implemented method is performed by an application comprising a testing tool, wherein the testing tool comprises a plurality of components, the plurality of components including a receiving component, an evaluation component, a scoring component, an execution component, and an output component;
wherein the test case bucket is received by a receiving component of the testing tool, wherein the respective test case is determined by an evaluation component of the testing tool, wherein the candidate score is incremented by a scoring component of the testing tool, wherein the execution is prioritized by an execution component of the testing tool, wherein an output of the first test case is output by an output component of the testing tool;
wherein the testing tool is operatively connected via a network to an integrated development environment (IDE) application, the software, and a repository service, wherein the testing tool, the IDE application, the software, and the repository service each execute on a respective, distinct computer, wherein the repository service is configured to provide access to a test case store, a code store, and a metadata store.

4. The computer-implemented method of claim 3 , wherein the code store maintains a plurality of different releases of the software, wherein the test case store provides automated tests for verifying each release of the software, the automated tests including functional verification tests, regression tests, code coverage measurement tests, and code execution verification tests;
wherein the metadata store maintains information describing files stored in the code store and the test case store, respectively, wherein the information includes patch descriptions, patch developers, modified code, modified files, targeted code areas, previous test results, and patch developers associated with failed test results.

5. The computer-implemented method of claim 1 , wherein the computer-implemented method is performed by a testing tool.

6. The computer-implemented method of claim 5 , wherein the testing tool includes at least two of a receiving component, an evaluation component, a scoring component, an execution component, and an output component.

7. The computer-implemented method of claim 5 , wherein the testing tool is operatively connected via a network to at least two of an integrated development environment (IDE) application, the software, and a repository service.

8. The computer-implemented method of claim 1 , wherein each test case further specifies at least one of a respective code coverage measure and a respective impact measure of the test case.

9. A computer program product to reduce a processing overhead incurred in testing areas of development code impacted by a target release of software, the computer program product comprising:
a computer-readable storage medium storing instructions which, when executed on one or more computer processors, performs one or more operations comprising:
receiving a test case bucket containing a plurality of test cases for executing on the target release of the software, wherein the target release includes one or more patches, each patch of which modifies a respective area of development code of a previous release of the software, wherein each test case is associated with a respective, candidate score and specifies a history of the respective test case relative to an execution on the previous release;
upon determining, for each of the one or more patches, a respective test case, of the plurality of test cases, that targets the area of development code modified by the respective patch, incrementing the candidate score associated with the respective test case;
for each patch, identifying one or more developers associated with the respective patch based on metadata of the respective patch;
evaluating a test case history associated with the respective one or more developers to determine existence of one or more test cases that have previously failed and that are associated with the respective one or more developers;
upon determining the existence of the one or more test cases, obtaining identifying information for the one or more test cases based on the metadata and incrementing a score for each of the one or more test cases, wherein the one or more test cases are included in the test case bucket; and
prioritizing, based on the score of each test case, execution of the test cases in the test case bucket during execution of the test case bucket on the target release of the software, wherein the test case having the highest score from the test case bucket is selected and executed first.

10. The computer program product of claim 9 , wherein each test case further specifies a respective code coverage measure and a respective impact measure of the test case.

11. The computer program product of claim 9 , wherein the one or more operations are performed by a testing tool.

12. The computer program product of claim 11 , wherein the testing tool includes at least two of a receiving component, an evaluation component, a scoring component, an execution component, and an output component.

13. The computer program product of claim 11 , wherein the testing tool is operatively connected via a network to at least two of an integrated development environment (IDE) application, the software, and a repository service.

14. The computer program product of claim 9 , wherein each test case further specifies at least one of a respective code coverage measure

Assignees

• IBM

Inventors

• Atyam Balaji V
• Bofferding Nicholas E
• Geissler Andrew J
• Hollinger Michael C
• Srr Sivas

Classifications

• G06F8/658

  Incremental updates; Differential updates · CPC title

• G06F11/3676 Primary

  for coverage analysis · CPC title

• G06F11/368 Primary

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/362

  Debugging of software · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title