Proactive cognitive analysis for inferring test case dependencies

Patent metadata

Field	Value
Publication number	US-10423519-B2
Application number	US-201715854257-A
Country	US
Kind code	B2
Filing date	Dec 26, 2017
Priority date	Jul 21, 2015
Publication date	Sep 24, 2019
Grant date	Sep 24, 2019

Abstract

Mechanisms are provided for evaluating test cases for testing a software product based on a requirements change. The mechanisms analyze a test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus and generate a test case relationship model based on the identified plurality of first relationships. The mechanisms receive a proposed requirements change to change one or more requirements of the software product and then perform a search of the test case relationship model to identify test case relationships corresponding to the proposed requirements change. The mechanisms identify a subset of test cases affected by the proposed requirements change and generate an output specifying the identified subset of test cases.

First claim

What is claimed is:

1. A method, in a data processing system comprising a processor and a memory, for evaluating test cases for testing a software product based on a requirements change, the method comprising:
analyzing, by the data processing system, a test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus;
generating, by the data processing system, a test case relationship model based on the identified plurality of first relationships;
receiving, by the data processing system, a proposed requirements change to change one or more requirements of the software product;
searching, by the data processing system, the test case relationship model to identify test case relationships corresponding to the proposed requirements change;
identifying, by the data processing system, a subset of test cases in the test case corpus affected by the proposed requirements change based on the identified test case relationships corresponding to the proposed requirements change at least by generating a ranked listing that ranks each test case having at least one test case relationship corresponding to the proposed requirements change, relative to other test cases having at least one test case relationship, based on whether the at least one test case relationship associated with the test case is a direct relationship or an indirect relationship, whether the test case has multiple test case relationships corresponding to the proposed requirements change, and whether the at least one test case relationship comprises both a direct relationship and an indirect relationship, and selecting test cases from the ranked listing to be the subset of test cases based on relative rankings of the test cases in the ranked listing; and
generating, by the data processing system, an output specifying the identified subset of test cases.

2. The method of claim 1 , wherein searching the test case relationship model to identify test case relationships corresponding to the proposed requirements change comprises identifying direct and indirect relationships between elements of the proposed requirements change and elements of relationships modeled in the test case relationship model.

3. The method of claim 1 , wherein analyzing the test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus, comprises:
analyzing each test case in the test case corpus to identify correspondence between inputs and outputs of each test case; and
generating the plurality of first relationships based on the correspondences between inputs and outputs of the test cases in the test case corpus.

4. The method of claim 1 , wherein analyzing the test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus comprises annotating the test cases based on identified test case attributes to facilitate identification of direct and indirect relationships between system attributes under test and test cases affected when system requirements are changed.

5. The method of claim 1 , further comprising performing natural language processing of a corpus of documentation referencing the software product to identify one or more second relationships associated with the software product, wherein generating a test case relationship model based on the identified plurality of first relationships further comprises generating the test case relationship model based on both the identified plurality of first relationships and the one or more second relationships.

6. The method of claim 5 , wherein generating the test case relationship model based on the identified plurality of first relationships and the one or more second relationships comprises scoring each of the plurality of first relationships and the one or more second relationships based on a search of supporting evidence in the test case corpus and the corpus of documentation for each of the plurality of first relationships and the one or more second relationships.

7. The method of claim 1 , wherein generating the output specifying the identified subset of test cases further comprises automatically re-executing test cases in the subset of test cases on the software product and generating an output of the re-executed test cases.

8. The method of claim 1 , wherein selecting test cases from the ranked listing to be the subset of test cases comprises:
comparing each test case's corresponding rank to at least one threshold rank value; and
selecting test cases from the ranked listing whose corresponding rank meets or exceeds the at least one threshold rank value.

9. The method of claim 8 , wherein, in response to the selected test cases comprising a null set of test cases due to no test cases in the ranked listing having corresponding ranks meeting or exceeding the at least one threshold rank value, generating the output specifying the identified subset of test cases further comprises identifying, in the output, a test case to be generated to test the software product with regard to the proposed requirements change.

10. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to:
analyze a test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus;
generate a test case relationship model based on the identified plurality of first relationships;
receive a proposed requirements change to change one or more requirements of the software product;
search the test case relationship model to identify test case relationships corresponding to the proposed requirements change;
identify a subset of test cases in the test case corpus affected by the proposed requirements change based on the identified test case relationships corresponding to the proposed requirements change at least by generating a ranked listing that ranks each test case having at least one test case relationship corresponding to the proposed requirements change, relative to other test cases having at least one test case relationship, based on whether the at least one test case relationship associated with the test case is a direct relationship or an indirect relationship, whether the test case has multiple test case relationships corresponding to the proposed requirements change, and whether the at least one test case relationship comprises both a direct relationship and an indirect relationship, and selecting test cases from the ranked listing to be the subset of test cases based on relative rankings of the test cases in the ranked listing; and
generate an output specifying the identified subset of test cases.

11. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to search the test case relationship model to identify test case relationships corresponding to the proposed requirements change at least by identifying direct and indirect relationships between elements of the proposed requirements change and elements of relationships modeled in the test case relationship model.

12. The computer program product of claim 10 , wherein the computer readable program further causes the data processing system to analyze the test case corpus to identify a plurality of first relationships between elements of test cases in the test case corpus, at least by:
analyzing each test case in the test case corpus to identify correspondence between inputs and outputs of each test case; and
generating

Assignees

• IBM

Inventors

• Andrejko Pamela D
• Freed Andrew R
• Salmon Richard A
• Skinner Charles S

Classifications

• G06F30/398

  Design verification or optimisation, e.g. using design rule check [DRC], layout versus schematics [LVS] or finite element methods [FEM] (optical proximity correction [OPC] design processes G03F1/36) · CPC title

• G06Q10/06

  Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F11/3672 Primary

  Test management · CPC title

• G06F8/43

  Checking; Contextual analysis · CPC title