Gray-box testing based on concrete usages

Patent metadata

Field	Value
Publication number	US-11074158-B2
Application number	US-201715828480-A
Country	US
Kind code	B2
Filing date	Dec 1, 2017
Priority date	Dec 1, 2017
Publication date	Jul 27, 2021
Grant date	Jul 27, 2021

Abstract

A computer implemented method for testing an application according to usage data includes receiving an application to be tested and a set of usage data corresponding to the application to be tested, wherein the set of usage data corresponds to previously executed code sequences, identifying one or more code sequences of interest corresponding to the received application, wherein the code sequences of interest correspond to codes sequences that are configured to exercise the received application, extracting concrete usages of the code sequence of interest from the received set of usage data, generating one or more test cases for the application according to the extracted usages, and providing the one or more generated test cases. The method may additionally include testing the application according to the one or more generated test cases.

First claim

What is claimed is:

1. A computer implemented method for testing an application according to usage data, the method comprising:
receiving an application to be tested and a set of usage data corresponding to the application to be tested, wherein the set of usage data is sourced from code repositories that make use of the received application;
identifying one or more code sequences of interest corresponding to the received application, wherein the code sequences of interest correspond to codes sequences that are configured to exercise the received application;
extracting concrete usages of the code sequence of interest from the received set of usage data, wherein the concrete usages correspond to actual usages of the one or more code sequences of interest;
identifying one or more parameters corresponding to the extracted concrete usages of the code sequence of interest;
generating one or more test cases for the application according to the extracted usages and the identified one or more parameters, wherein the one or more test cases test combinations of the identified parameters to determine if an application failure is caused by one or more unique combinations of parameters; and
outputting the one or more generated test cases.

2. The computer implemented method of claim 1 , further comprising identifying one or more dependencies corresponding to the extracted concrete usages of the code sequence of interest.

3. The computer implemented method of claim 2 , wherein generating one or more test cases for the application according to the extracted usages includes generating test cases configured to test the application with respect to the identified one or more dependencies corresponding to the extracted concrete usages.

4. The computer implemented method of claim 1 , wherein generating one or more test cases for the application according to the extracted usages includes generating test cases configured to test the application with respect to the identified one or more parameters corresponding to the extracted concrete usages.

5. The computer implemented method of claim 1 , wherein identifying one or more code sequences of interest includes searching for invocations executed with respect to the received application.

6. The computer implemented method of claim 1 , wherein providing the one or more generated test cases includes generating a report that indicates a set of parameters or dependencies being tested by each generated test case.

7. The computer implemented method of claim 1 , further comprising executing the generated test according to the one or more generated test cases.

8. The method of claim 1 , wherein the one or more test cases test parameter value ranges corresponding to the identified one or more parameters to determine if a unique range of parameters causes an application failure.

9. The method of claim 1 , wherein the one or more test cases test dependencies among methods or endpoints according to the identified one or more parameters.

10. The method of claim 9 , wherein generating one or more test cases further includes:
determining that a method or end point can have up to “n” parameters; and
generating test cases to test combinations of the n parameters to determine if there is an application failure caused by one or more unique combinations.

11. A computer program product for testing an application according to usage data, the computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising instructions to:
receive an application to be tested and a set of usage data corresponding to the application to be tested, wherein the set of usage data is sourced from code repositories that make use of the received application;
identify one or more code sequences of interest corresponding to the received application, wherein the code sequences of interest correspond to codes sequences that are configured to exercise the received application;
extract concrete usages of the code sequence of interest from the received set of usage data, wherein the concrete usages correspond to actual usages of the one or more code sequences of interest;
identify one or more parameters corresponding to the extracted concrete usages of the code sequence of interest;
generate one or more test cases for the application according to the extracted usages and the identified one or more parameters, wherein the one or more test cases test combinations of the identified parameters to determine if an application failure is caused by one or more unique combinations of parameters; and
output the one or more generated test cases.

12. The computer program product of claim 11 , further comprising instructions to identify one or more dependencies corresponding to the extracted concrete usages of the code sequence of interest.

13. The computer program product of claim 11 , wherein generating one or more test cases for the application according to the extracted usages includes generating test cases configured to test the application with respect to the identified one or more parameters corresponding to the extracted concrete usages.

14. The computer program product of claim 11 , wherein instructions to identify one or more code sequences of interest comprise instructions to search for invocations executed with respect to the received application.

15. The computer program product of claim 11 , wherein instructions to provide the one or more generated test cases comprise instructions to generate a report that indicates a set of parameters or dependencies being tested by each generated test case.

16. A computer system for testing an application according to usage data, the computer program product comprising:
one or more computer processors;
one or more computer readable storage media;
program instructions stored on the one or more computer readable storage media, the program instructions comprising instructions to:
receive an application to be tested and a set of usage data corresponding to the application to be tested, wherein the set of usage data is sourced from code repositories that make use of the received application;
identify one or more code sequences of interest corresponding to the received application, wherein the code sequences of interest correspond to codes sequences that are configured to exercise the received application;
extract concrete usages of the code sequence of interest from the received set of usage data, wherein the concrete usages correspond to actual usages of the one or more code sequences of interest;
identify one or more parameters corresponding to the extracted concrete usages of the code sequence of interest;
generate one or more test cases for the application according to the extracted usages and the identified one or more parameters, wherein the one or more test cases test combinations of the identified parameters to determine if an application failure is caused by one or more unique combinations of parameters; and
output the one or more generated test cases.

17. The computer system of claim 16 , further comprising instructions to identify one or more dependencies corresponding to the extracted concrete usages of the code sequence of interest.

18. The computer system of claim 16 , wherein generating one or more test cases for the application according to the extracted usages includes generating test cases configured to test the application with respect to the identified one or more parameters correspo

Assignees

• IBM

Inventors

• Laredo Jim A
• Slominski Aleksander
• Wittern John E
• Ying Annie T
• Young Christopher C
• Zheng Yunhui

Classifications

• G06F11/3684 Primary

  for test design, e.g. generating new test cases · CPC title

• G06F8/433

  Dependency analysis; Data or control flow analysis · CPC title

• G06F11/3414

  Workload generation, e.g. scripts, playback · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title