Adaptive framework automatically prioritizing software test cases

Patent metadata

Field	Value
Publication number	US-9489289-B2
Application number	US-201414565220-A
Country	US
Kind code	B2
Filing date	Dec 9, 2014
Priority date	Dec 9, 2014
Publication date	Nov 8, 2016
Grant date	Nov 8, 2016

Abstract

An automated, self-adaptive framework prioritizes software testing in a consistent and effective manner. A metric evaluates past test execution information for assigning regression testing priority. The metric may be calculated with reference to one or more of the following factors taken in combination: requirement, coverage, history, and cost. The requirement factor considers customer-assigned priority of testing the code, complexity of implementing the code, and proneness of the code to faults. The coverage factor considers code coverage, feature coverage, and common usage rate. The history factor considers previous bug found rate, case stable rate, and priority to calculate. The cost factor considers test case execution time, and step length. A value of each factor for one test case is measured according to that test case and is not related to other test cases. The calculation result representing the metric for each test case determines a priority of the test case.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving a user input and information about a software test case from a database, wherein the information comprises coverage information including a code coverage rate and a separate feature coverage rate calculated from a keyword;
processing the information to calculate in a first testing round, a first metric for the software test case, the first metric comprising a weighted code coverage factor and a weighted feature coverage factor;
assigning a testing priority to the software test case for the first testing round based upon the first metric; and
communicating the testing priority to the database for storage.

2. A method as in claim 1 further comprising:
receiving another user input and updated information about the software test case from the database;
processing the updated information to calculate in a second testing round, a second metric for the software test case;
assigning an adaptive testing priority to the software test case for the second testing round based on the second metric; and
communicating the adaptive testing priority to the database for storage.

3. A method as in claim 2 comprising:
receiving the testing priority from the database; and
calculating the second metric based upon the testing priority and the updated information.

4. A method as in claim 2 wherein the updated information comprises updated history information, updated coverage information, and updated cost information.

5. A method as in claim 1 wherein the information comprises requirement information including a customer-assigned priority value, an implementation complexity value, and a fault proneness value.

6. A method as in claim 1 wherein the information comprises coverage information further including a common usage rate separate from the code coverage rate and the feature coverage rate.

7. A method as in claim 1 wherein the information comprises history information including a case stable rate and a bug found rate.

8. A method as in claim 1 wherein the information comprises cost information including a step length rate and a test case execution time rate.

9. A method as in claim 1 further comprising communicating the first metric to the database for storage.

10. A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising:
receiving a user input and information about a software test case from a database, the information comprising coverage information including a code coverage rate and a separate feature coverage rate calculated from a keyword;
processing the information to calculate in a first testing round, a first metric for the software test case, the first metric comprising a weighted code coverage factor and a weighted feature coverage factor;
assigning a testing priority to the software test case for the first testing round based upon the first metric; and
communicating the testing priority to the database for storage.

11. A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises:
receiving another user input and updated information about the software test case from the database;
processing the updated information to calculate in a second testing round, a second metric for the software test case;
assigning an adaptive testing priority to the software test case for the second testing round based on the second metric; and
communicating the adaptive testing priority to the database for storage.

12. A non-transitory computer readable storage medium as in claim 11 wherein the method further comprises:
receiving the testing priority from the database; and
calculating the second metric based upon the testing priority and the updated information.

13. A non-transitory computer readable storage medium as in claim 12 wherein the updated information comprises updated history information, updated coverage information, and updated cost information.

14. A non-transitory computer readable storage medium as in claim 10 wherein the information further comprises at least one of:
requirement information including a customer-assigned priority value, an implementation complexity value, and a fault proneness value;
the coverage information further including a common usage rate separate from the code coverage rate and the feature coverage rate;
history information including a case stable rate and a bug found rate; and
cost information including a step length rate and a test case execution time rate.

15. A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises communicating the first metric to the database for storage.

16. A computer system comprising:
one or more hardware processors;
a software program, stored in a non-transitory computer-readable storage medium and executable on one of said hardware processors, the software program configured to:
cause a prioritization engine stored in the non-transitory computer-readable storage medium to receive a user input and information about a software test case from a database stored in the non-transitory computer-readable storage medium, wherein the information comprises coverage information including a code coverage rate and a separate feature coverage rate calculated from a keyword;
cause the prioritization engine to process the information to calculate in a first testing round, a first metric for the software test case, the first metric comprising a weighted code coverage factor and a weighted feature coverage factor;
cause the prioritization engine to assign a testing priority to the software test case for the first testing round based upon the first metric;
cause the prioritization engine to communicate the testing priority to the database for storage;
cause the prioritization engine to receive another user input and updated information about the software test case from the database;
cause the prioritization engine to process the updated information to calculate in a second testing round, a second metric for the software test case;
cause the prioritization engine to assign an adaptive testing priority to the software test case for the second testing round based on the second metric; and
cause the prioritization engine to communicate the adaptive testing priority to the database for storage.

17. A computer system as in claim 16 wherein:
the prioritization engine receives the testing priority from the database; and
the prioritization engine calculates the second metric based upon the testing priority and the updated information.

18. A computer system as in claim 17 wherein the updated information further comprises updated coverage information, updated history information and updated cost information.

19. A computer system as in claim 16 wherein the information comprises at least one of:
requirement information including a customer-assigned priority value, an implementation complexity value, and a fault proneness value;
the coverage information further including a common usage rate separate from the code coverage rate and the feature coverage rate;
history information including a case stable rate and a bug found rate; and
cost information including a step length rate and a test case execution time rate.

20. A computer system as in claim 16 wherein the software program further comprises code configured to cause the prioritization engine to communicate the first metric to the database for storage.

Assignees

• Hu Ziheng
• Li Jing
• Li Xin
• Zhou Chen
• Chang Longhui
• Sap Se

Inventors

• Hu Ziheng
• Li Jing
• Li Xin
• Zhou Chen
• Chang Longhui

Classifications

• G06F11/3684 Primary

  for test design, e.g. generating new test cases · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title