Mutation testing in parallel threads

Patent metadata

Field	Value
Publication number	US-2022327043-A1
Application number	US-202117486682-A
Country	US
Kind code	A1
Filing date	Sep 27, 2021
Priority date	Apr 7, 2021
Publication date	Oct 13, 2022
Grant date	—

Abstract

Mutation testing can indicate whether mutants of a software application, created by intentionally altering source code of the software application, are successfully “killed” by test cases executed against the mutants. Mutation testing can be performed via parallel threads by, within each parallel thread, modifying individual source code class files and recompiling the modified class files to generate and test mutants. Individual mutation test results produced within each of the parallel threads can be aggregated to generate an aggregated test result report that indicates overall testing metrics associated with the mutation testing across the parallel threads.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
initiating, by one or more processors of one or more computing devices, a plurality of parallel threads; identifying, by the one or more processors, and in different threads of the plurality of parallel threads, mutatable source code class files associated with a software application; modifying, by the one or more processors, and in the different threads, the mutatable source code class files based on at least one mutation configuration; compiling, by the one or more processors, and in the different threads, the mutatable source code class files into compiled bytecode; and executing, by the one or more processors, and in the different threads, a plurality of test cases against application mutants that include the compiled bytecode.

2 . The computer-implemented method of claim 1 , wherein identifying the mutatable source code class files comprises:
determining, by the one or more processors, code coverage associated with a plurality of source class files by executing the plurality of test cases against an unmodified version of the software application; and identifying, by the one or more processors, and in the different threads, source class files that:
have at least a threshold level of the code coverage; and
correspond to the at least one mutation configuration.

3 . The computer-implemented method of claim 1 , wherein the application mutants include previously compiled bytecode associated with unmodified source code files, and the compiled bytecode.

4 . The computer-implemented method of claim 3 , wherein executing the plurality of test cases against the application mutants comprises swapping, in memory associated with the different threads, second previously compiled bytecode with the compiled bytecode without recompiling the previously compiled bytecode.

5 . The computer-implemented method of claim 1 , wherein an individual thread, of the different threads, is associated with multiple threads and uses the multiple threads to execute different test cases, of the plurality of test cases, against a same application mutant concurrently.

6 . The computer-implemented method of claim 1 , further comprising:
logging, by the one or more processors, and in the different threads, mutation test results associated with executing the plurality of test cases against the application mutants; and aggregating, by the one or more processors, the mutation test results into an aggregated test result report.

7 . The computer-implemented method of claim 6 , wherein the aggregated test result report includes:
summary mutation testing information associated with the different threads; and mutation testing details associated with the mutatable source code class files.

8 . One or more computing devices, comprising:
one or more processors; memory storing:
application source code, associated with a software application, comprising a plurality of class files;
a plurality of test cases associated with the software application;
one or more mutation configurations; and
computer-executable instructions associated with a mutation test manager that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
initiating a plurality of parallel threads;
distributing the plurality of class files among the plurality of parallel threads;
identifying, in the plurality of parallel threads, mutatable class files, of the plurality of class files, based on the one or more mutation configurations;
modifying, in the plurality of parallel threads, the mutatable class files based on the one or more mutation configurations;
compiling, in the plurality of parallel threads, the mutatable class files into compiled bytecode; and
executing, in the plurality of parallel threads, at least a portion of the plurality of test cases against a plurality of application mutants that include the compiled bytecode.

9 . The one or more computing devices of claim 8 , wherein identifying the mutatable class files comprises:
identifying class files, of the plurality of class files, that include at least a threshold percentage of source code lines that are covered by the plurality of test cases; and determining that the source code lines of the class files correspond to at least one mutation configuration of the one or more mutation configurations.

10 . The one or more computing devices of claim 8 , wherein:
the plurality of class files is organized into a plurality of class file packages, distributing the plurality of class files comprises distributing the plurality of class file packages among the plurality of parallel threads, and identifying the mutatable class files comprises identifying at least one mutatable class file within a set of class file packages associated with an individual parallel thread.

11 . The one or more computing devices of claim 8 , wherein the plurality of application mutants include previously compiled bytecode associated with unmodified source code files of the plurality of class files, and the compiled bytecode.

12 . The one or more computing devices of claim 11 , wherein executing the at least the portion of the plurality of test cases against the plurality of application mutants comprises swapping, in the memory, second previously compiled bytecode with the compiled bytecode, without recompiling the previously compiled bytecode.

13 . The one or more computing devices of claim 8 , wherein an individual thread, of the plurality of parallel threads, is associated with multiple threads and is configured to use the multiple threads to execute different test cases, of the plurality of test cases, against a same application mutant concurrently.

14 . The one or more computing devices of claim 8 , wherein the operations further comprise:
logging a plurality of mutation test results associated with executing the at least the portion of the plurality of test cases against the plurality of application mutants in the plurality of parallel threads; and aggregating the plurality of mutation test results into an aggregated test result report.

15 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to:
initiate a first parallel thread and a second parallel thread; and perform, substantially concurrently:
first parallel thread operations, associated with the first parallel thread, comprising:
identifying a first mutatable source code class file associated with a software application;
modifying the first mutatable source code class file based on at least one mutation configuration;
compiling the first mutatable source code class file into one or more first compiled bytecode class files; and
executing a plurality of test cases against a first application mutant that includes the one or more first compiled bytecode class files; and

second parallel thread operations, associated with the second parallel thread, comprising:
identifying a second mutatable source code class file associated with the software application;
modifying the second mutatable source code class file based on the at least one mutation configuration;
compiling the second mutatable source code class file into one or more second compiled bytecode class files; and
executing the plurality of test cases against a second application mutant that includes the one or more second compiled bytecode class files.

16 . The one or more non-transitory computer-readab

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Pearson Andrew L
• Shepherd Nate

Classifications

• G06F8/44

  Encoding · CPC title

• G06F9/45516

  Runtime code conversion or optimisation · CPC title

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title