Mutation testing in parallel threads

Patent metadata

Field	Value
Publication number	US-11797429-B2
Application number	US-202117486682-A
Country	US
Kind code	B2
Filing date	Sep 27, 2021
Priority date	Apr 7, 2021
Publication date	Oct 24, 2023
Grant date	Oct 24, 2023

Abstract

Mutation testing can indicate whether mutants of a software application, created by intentionally altering source code of the software application, are successfully “killed” by test cases executed against the mutants. Mutation testing can be performed via parallel threads by, within each parallel thread, modifying individual source code class files and recompiling the modified class files to generate and test mutants. Individual mutation test results produced within each of the parallel threads can be aggregated to generate an aggregated test result report that indicates overall testing metrics associated with the mutation testing across the parallel threads.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
initiating, by one or more processors of one or more computing devices, a plurality of parallel threads;
identifying, by the one or more processors, and in different threads of the plurality of parallel threads, mutatable source code class files associated with a software application;
generating, by the one or more processors, and in the different threads, mutated source code class files by modifying the mutatable source code class files based on at least one mutation configuration;
compiling, by the one or more processors, and in the different threads, the mutated source code class files into a different amount of compiled bytecode class files; and
executing, by the one or more processors, and in the different threads, a plurality of test cases against application mutants that include the compiled bytecode class files.

2. The computer-implemented method of claim 1 , wherein identifying the mutatable source code class files comprises:
determining, by the one or more processors, code coverage associated with a plurality of source class files by executing the plurality of test cases against an unmodified version of the software application; and
identifying, by the one or more processors, and in the different threads, source class files that:
have at least a threshold level of the code coverage; and
correspond to the at least one mutation configuration.

3. The computer-implemented method of claim 1 , wherein the application mutants include:
previously compiled bytecode class files associated with unmodified source code class files, and
the compiled bytecode class files associated with the mutated source code class files.

4. The computer-implemented method of claim 3 , wherein executing the plurality of test cases against the application mutants comprises swapping, in memory associated with the different threads, second previously compiled bytecode class files with the compiled bytecode class files without recompiling the previously compiled bytecode class files.

5. The computer-implemented method of claim 1 , wherein an individual thread, of the different threads, is associated with multiple threads and uses the multiple threads to execute different test cases, of the plurality of test cases, against a same application mutant concurrently.

6. The computer-implemented method of claim 1 , further comprising:
logging, by the one or more processors, and in the different threads, mutation test results associated with executing the plurality of test cases against the application mutants; and
aggregating, by the one or more processors, the mutation test results into an aggregated test result report.

7. The computer-implemented method of claim 6 , wherein the aggregated test result report includes:
summary mutation testing information associated with the different threads; and
mutation testing details associated with the mutatable source code class files.

8. One or more computing devices, comprising:
one or more processors;
memory storing:
application source code, associated with a software application, comprising a plurality of source code class files;
a plurality of test cases associated with the software application;
one or more mutation configurations; and
computer-executable instructions associated with a mutation test manager that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
initiating a plurality of parallel threads;
distributing the plurality of source code class files among the plurality of parallel threads;
identifying, in the plurality of parallel threads, mutatable source code class files, of the plurality of source code class files, based on the one or more mutation configurations;
generating, in the plurality of parallel threads, mutated source code class files by modifying the mutatable source code class files based on the one or more mutation configurations;
compiling, in the plurality of parallel threads, the mutated source code class files into a different amount of compiled bytecode class files; and
executing, in the plurality of parallel threads, at least a portion of the plurality of test cases against a plurality of application mutants that include the compiled bytecode class files.

9. The one or more computing devices of claim 8 , wherein identifying the mutatable source code class files comprises:
identifying source code class files, of the plurality of source code class files, that include at least a threshold percentage of source code lines that are covered by the plurality of test cases; and
determining that the source code lines of the source code class files correspond to at least one mutation configuration of the one or more mutation configurations.

10. The one or more computing devices of claim 8 , wherein:
the plurality of source code class files is organized into a plurality of class file packages,
distributing the plurality of source code class files comprises distributing the plurality of class file packages among the plurality of parallel threads, and
identifying the mutatable source code class files comprises identifying at least one mutatable source code class file within a set of class file packages associated with an individual parallel thread.

11. The one or more computing devices of claim 8 , wherein the plurality of application mutants include:
previously compiled bytecode class files associated with unmodified source code class files of the plurality of source code class files, and
the compiled bytecode class files associated with the mutated source code class files.

12. The one or more computing devices of claim 11 , wherein executing the at least the portion of the plurality of test cases against the plurality of application mutants comprises swapping, in the memory, second previously compiled bytecode class files with the compiled bytecode class files, without recompiling the previously compiled bytecode class files.

13. The one or more computing devices of claim 8 , wherein an individual thread, of the plurality of parallel threads, is associated with multiple threads and is configured to use the multiple threads to execute different test cases, of the plurality of test cases, against a same application mutant concurrently.

14. The one or more computing devices of claim 8 , wherein the operations further comprise:
logging a plurality of mutation test results associated with executing the at least the portion of the plurality of test cases against the plurality of application mutants in the plurality of parallel threads; and
aggregating the plurality of mutation test results into an aggregated test result report.

15. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to:
initiate a first parallel thread and a second parallel thread; and
perform, substantially concurrently:
first parallel thread operations, associated with the first parallel thread, comprising:
identifying a first mutatable source code class file associated with a software application;
generating a first mutated source code class file by modifying the first mutatable source code class file based on at least one mutation configuration;
compiling the first mutated source code class file into two or more first compiled bytecode class files; and
executing a plurality of test cases against a first application mutant that includes the two or more first compiled bytecode class files; and

second parallel th

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Pearson Andrew L
• Shepherd Nate

Classifications

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F8/30

  Creation or generation of source code · CPC title

• G06F8/44

  Encoding · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06F11/3692

  for test results analysis · CPC title