Identifying test gaps using code execution paths

Patent metadata

Field	Value
Publication number	US-10241897-B2
Application number	US-201615361895-A
Country	US
Kind code	B2
Filing date	Nov 28, 2016
Priority date	May 28, 2014
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

Systems and techniques are described for identifying test gaps. A described technique includes identifying production code paths for an application. Each production code path specifies a respective sequence of code of the application that was executed in a production environment. Test code paths are identified for the application. Each test code path specifies a respective sequence of the application that was tested in a test environment. The production code paths are compared to the test code paths to identify a set of test gaps for the application. Each test gap specifies a respective production code path that is not included in the test code paths. Test gap data specifying the test gaps for the application can be provided for presentation to a user.

First claim

What is claimed is:

1. A computer-implemented method comprising:
identifying, by at least one computing device, a plurality of production code paths for an application, each of the plurality of production code paths specifying a respective sequence of code of the application that was executed in a production environment;
identifying, by the at least one computing device, a plurality of test code paths for the application, each of the plurality of test code paths specifying a respective sequence of code of the application that was tested in a test environment;
comparing, by the at least one computing device, the plurality of production code paths to the plurality of test code paths to identify at least one test gap for the application, the at least one test gap specifying at least one of the plurality of production code paths that is not included in the plurality of test code paths; and
displaying, by the at least one computing device, test gap data specifying the at least one test gap for the application.

2. The method of claim 1 , further comprising modifying, by the at least one computing device, test code executed in the test environment to cover the at least one test gap for the application.

3. The method of claim 1 , wherein the plurality of production code paths is identified based at least in part on log data generated during execution of the application in the production environment.

4. The method of claim 1 , further comprising:
receiving, by the at least one computing device, production data associated with the application from a plurality of user devices; and
storing, by the at least one computing device, the production data, wherein the plurality of production code paths is identified based at least in part on the production data received from the plurality of user devices.

5. The method of claim 1 , further comprising
monitoring, by the at least one computing device, a test of the application executing in the test environment; and
logging, by the at least one computing device, data specifying a sequence of code executed during the data logged during the test.

6. The method of claim 1 , further comprising
identifying, by the at least one computing device, a plurality of potential code paths for the application by analyzing source code of the application, each of the plurality of potential code paths specifying a respective sequence of code of the application; and
comparing, by the at least one computing device, the plurality of potential code paths to the plurality of test code paths to identify at least one second test gap for the application, wherein the test gap data further comprises the at least one second test gap.

7. The method of claim 1 , further comprising:
generating, by the at least one computing device, a first tree data structure that specifies the plurality of production code paths; and
generating, by the at least one computing device, a second tree data structure that specifies the plurality of test code paths, wherein comparing the plurality of production code paths to the plurality of test code paths comprises comparing the first tree data structure to the second tree data structure.

8. A system comprising:
a data store comprising an application; and
at least one computing device in communication with the data store, the at least one computing device configured to at least
identify a plurality of production code paths for the application, each of the plurality of production code paths specifying a respective sequence of code of the application that was executed in a production environment;
identify a plurality of test code paths for the application, each of the plurality of test code paths specifying a respective sequence of code of the application that was tested in a test environment;
compare the plurality of production code paths to the plurality of test code paths to identify at least one test gap for the application, the at least one test gap specifying at least one of the plurality of production code paths that is not included in the plurality of test code paths; and
display test gap data specifying the at least one test gap for the application.

9. The system of claim 8 , wherein the at least one computing device is further configured to at least modify test code executed in the test environment to cover the at least one test gap for the application.

10. The system of claim 8 , wherein the plurality of production code paths is identified based at least in part on log data generated during execution of the application in the production environment.

11. The system of claim 8 , wherein the at least one computing device is further configured to at least:
receive production data associated with the application from a plurality of user devices; and
store the production data, wherein the plurality of production code paths is identified based at least in part on the production data received from the plurality of user devices.

12. The system of claim 8 , wherein the at least one computing device is further configured to at least:
monitor a test of the application executing in the test environment; and
log data specifying a sequence of code executed during the data logged during the test.

13. The system of claim 8 , wherein the at least one computing device is further configured to at least:
identify a plurality of potential code paths for the application by analyzing source code of the application, each of the plurality of potential code paths specifying a respective sequence of code of the application; and
compare the plurality of potential code paths to the plurality of test code paths to identify at least one second test gap for the application, wherein the test gap data further comprises the at least one second test gap.

14. The system of claim 8 , wherein the at least one computing device is further configured to at least:
generate a first tree data structure that specifies the plurality of production code paths;
generate a second tree data structure that specifies the plurality of test code paths; and
compare the first tree data structure to the second tree data structure.

15. A non-transitory computer-readable medium embodying a program that, when executed by at least one computing device, causes the at least one computing device to at least:
identify a plurality of production code paths for an application, each of the plurality of production code paths specifying a respective sequence of code of the application that was executed in a production environment;
identify a plurality of test code paths for the application, each of the plurality of test code paths specifying a respective sequence of code of the application that was tested in a test environment;
compare the plurality of production code paths to the plurality of test code paths to identify at least one test gap for the application, the at least one test gap specifying at least one of the plurality of production code paths that is not included in the plurality of test code paths; and
display test gap data specifying the at least one test gap for the application.

16. The non-transitory computer-readable medium of claim 15 , wherein the program further causes the at least one computing device to at least modify test code executed in the test environment to cover the at least one test gap for the application.

17. The non-transitory computer-readable medium of claim 15 , wherein the plurality of production code paths is identified based at least in part on log data generated during execution of the application in the production environment.

Assignees

• Vmware Inc

Inventors

• Sinha Vineet Kumar
• Sarkar Prasenjit
• Kumar Prashant
• Shukla Anoop
• T K Sree Ranga Sudha

Classifications

• G06F11/3664

  Physics · mapped topic

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F11/3668

  Testing of software · CPC title

• G06F11/3672

  Test management · CPC title

• G06F11/3676 Primary

  for coverage analysis · CPC title