Application testing using multiple context-aware threads

Patent metadata

Field	Value
Publication number	US-10387295-B1
Application number	US-201514704886-A
Country	US
Kind code	B1
Filing date	May 5, 2015
Priority date	May 5, 2015
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

Technology is described for testing applications. Multiple testing threads that are to be used for testing an application may be identified. The multiple testing threads may be dispatched for execution using a unit testing framework. The multiple testing threads may be executed in parallel to test the application using testing context information that is related to each testing thread. Performance metrics may be collected for the application that is tested. The performance metrics may be transmitted from the multiple testing threads after parallel execution of the multiple testing threads.

First claim

What is claimed is:

1. A method, comprising:
identifying testing threads that are to perform testing of an application; and
launching the testing threads for parallel execution based, at least in part, on testing context information associated with each testing thread, wherein the testing context information includes a list of pre-computations to be performed before execution of the testing thread, and a list of tear downs to be performed before execution of the testing thread.

2. The method of claim 1 , further comprising:
receiving performance metrics for each testing thread after the parallel execution of the testing threads, the performance metrics based at least in part on a load placed on portions of the application tested in the service provider environment; and
aggregating the performance metrics for the application that is tested in order to provide a report of application test metrics.

3. The method of claim 1 , wherein the testing context information is independent for each testing thread that is being launched from a test runner that operates in a unit testing framework of the service provider environment.

4. The method of claim 1 , wherein the testing context information includes a test name associated with the testing thread to be executed.

5. The method of claim 1 , wherein the testing threads are initialized using the testing context information by a thread pool dispatcher in a test runner prior to being launched for parallel execution on an execution server.

6. A method for testing applications, the method comprising:
identifying a plurality of testing threads that are to be used for testing an application;
dispatching the plurality of testing threads for execution, the plurality of testing threads executing concurrently to test the application using testing context information that is linked to each testing thread, wherein the testing context information is locally maintained at each of the plurality of testing threads, and the testing context information for each testing thread includes a list of pre-computations to be performed before execution of the testing thread, and a list of tear downs to be performed before execution of the testing thread; and
collecting performance metrics for the application that is tested, the performance metrics being transmitted from the plurality of testing threads during execution of the plurality of testing threads.

7. The method of claim 6 , further comprising reporting the performance metrics for the application that is tested to identify problems in the application.

8. The method of claim 6 , wherein the plurality of testing threads are executed concurrently for testing the application using a test runner that operates using a unit testing framework.

9. The method of claim 6 , wherein the testing context information is locally maintained for each testing thread using a thread context manager in a unit testing framework.

10. The method of claim 6 , wherein the testing context information for each testing thread includes a test name associated with the testing thread to be executed and test data.

11. The method of claim 6 , wherein separate testing context information is linked to each testing thread in order to reduce a likelihood of race conditions occurring during testing of the application.

12. The method of claim 6 , further comprising maintaining a pool of testing threads, wherein a single testing thread in the pool is dispatched for execution using a unit testing framework in order to test a portion of the application.

13. The method of claim 6 , wherein status information for each testing thread that has been dispatched for execution is stored by a unit testing framework.

14. The method of claim 6 , wherein testing threads corresponding to portions of an application waiting to be tested are initially included in a testing thread pool queue, the testing threads being subsequently moved from the testing thread pool queue to a pool of testing threads that are to be launched for execution using a unit testing framework.

15. The method of claim 6 , further comprising incorporating one or more check points during application testing to determine an execution time for executing portions of the plurality of testing threads.

16. A system for testing an application in a service provider environment, the system comprising:
a processor;
a memory device including a data store to store a plurality of data and instructions that, when executed by the processor, cause the processor to:
create a pool of testing threads, wherein each testing thread in the pool is associated with a portion of the application to be tested;
initiate multiple testing threads in the pool for parallel execution using a test runner that operates in a unit testing framework, the multiple testing threads executing in parallel using testing context information that is linked to each testing thread, wherein the testing context information includes a list of pre-computations to be performed before execution of the testing thread, and a list of tear downs to be performed before execution of the testing thread; and
receive performance metrics from each testing thread to indicate operational data for the portion of the application that is tested in the service provider environment.

17. The system of claim 16 , wherein the plurality of data and instructions, when executed by the processor, cause the processor to: aggregate the performance metrics for the application that is tested in order to provide a report of application test metrics.

18. The system of claim 16 , wherein the pool of testing threads is a fixed pool that initiates a defined number of testing threads for execution at a time, wherein testing threads that are waiting to be executed are included in a testing thread pool queue before being added to the pool of testing threads.

19. The system of claim 16 , wherein the multiple testing threads are executed in parallel for testing the application using a test runner that operates using the unit testing framework.

20. The system of claim 16 , wherein status information for each testing thread that has been initiated is stored by the unit testing framework.

Assignees

• Amazon Tech Inc

Inventors

• Kesarwani Nitin

Classifications

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3692

  for test results analysis · CPC title

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title