Mutation testing in parallel threads

Patent metadata

Field	Value
Publication number	US-12174733-B2
Application number	US-202318466756-A
Country	US
Kind code	B2
Filing date	Sep 13, 2023
Priority date	Apr 7, 2021
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

Mutation testing can indicate whether mutants of a software application, created by intentionally altering source code of the software application, are successfully “killed” by test cases executed against the mutants. Mutation testing can be performed via parallel threads by, within each parallel thread, modifying individual source code class files and recompiling the modified class files to generate and test mutants. Individual mutation test results produced within each of the parallel threads can be aggregated to generate an aggregated test result report that indicates overall testing metrics associated with the mutation testing across the parallel threads.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
identifying, by one or more processors, mutatable source code class files associated with a software application;
generating, by the one or more processors, mutated source code class files by modifying the mutatable source code class files based on at least one mutation configuration;
compiling, by the one or more processors, the mutated source code class files into compiled bytecode class files that have a different structure than an original structure of the mutated source code class files;
generating, by the one or more processors, application mutants by swapping first previously-compiled bytecode files with one or more of the compiled bytecode class files in memory, without recompiling additional second previously-compiled bytecode files; and
executing, by the one or more processors, and in parallel, test cases against the application mutants.

2. The computer-implemented method of claim 1 , wherein:
the original structure of the mutated source code class files is characterized by a first number of files, and
the different structure of the compiled bytecode class files is characterized by a second number of files that is different from the first number of files.

3. The computer-implemented method of claim 1 , wherein:
the original structure of the mutated source code class files is characterized by a first number of classes, and
the different structure of the compiled bytecode class files is characterized by a second number of classes that is different from the first number of classes.

4. The computer-implemented method of claim 1 , wherein instances of at least one of generating the mutated source code class files or compiling the mutated source code class files are performed in parallel.

5. The computer-implemented method of claim 1 , wherein executing the test cases against the application mutants comprises executing, in parallel, the test cases against different application mutants, the different application mutants comprising different sets of the compiled bytecode class files that are characterized by different modifications to the mutatable source code class files.

6. The computer-implemented method of claim 1 , wherein executing the test cases against the application mutants comprises executing different sets of test cases, concurrently and in parallel, against a particular application mutant, of the application mutants.

7. The computer-implemented method of claim 1 , wherein identifying the mutatable source code class files comprises:
determining, by the one or more processors, code coverage metrics associated with a plurality of source class files by executing the test cases against an unmodified version of the software application; and
identifying, by the one or more processors, and based on the code coverage metrics, source class files that have at least a threshold code coverage level and correspond to the at least one mutation configuration.

8. One or more computing devices, comprising:
one or more processors; and
first memory storing computer-executable instructions associated with a mutation test manager that, when executed by the one or more processors, cause the one or more processors to:
generate mutated source code class files by modifying original source code class files, associated with a software application, based on at least one mutation configuration;
compile the mutated source code class files into compiled bytecode class files that have a different structure than an original structure of the original source code class files;
generate application mutants by swapping first previously-compiled bytecode files with one or more of the compiled bytecode class files in the first memory or a second memory, without recompiling additional second previously-compiled bytecode files; and
execute, in parallel, test cases against the application mutants.

9. The one or more computing devices of claim 8 , wherein:
a first mutated source code class file, of the mutated source code class files, comprises a single mutated class, and
the compiled bytecode class files comprise two or more classes that correspond to the single mutated class.

10. The one or more computing devices of claim 8 , wherein the computer-executable instructions cause the one or more processors to perform at least one of generating the mutated source code class files in parallel or compiling the mutated source code class files in parallel.

11. The one or more computing devices of claim 10 , wherein different application mutants are generated by swapping different first previously-compiled bytecode files with different compiled bytecode class files.

12. The one or more computing devices of claim 8 , wherein:
the computer-executable instructions cause the one or more processors to:
identify particular source code class files, of a plurality of source code class files, that include at least a threshold percentage of source code lines that are covered by the test cases; and
determine that the source code lines of the particular source code class files correspond to the at least one mutation configuration, and
the original source code class files, modified based on the at least one mutation configuration, comprise the particular source code class files.

13. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to:
generate mutated source code class files by modifying original source code class files, associated with a software application, based on at least one mutation configuration;
compile the mutated source code class files into compiled mutated bytecode class files that have a different structure than an original structure of the original source code class files;
generate different application mutants by swapping first sets of previously-compiled bytecode files with different sets of one or more of the compiled mutated bytecode class files, without recompiling other previously-compiled bytecode files; and
perform mutation testing by executing, in parallel, test cases against the different application mutants.

14. The one or more non-transitory computer-readable media of claim 13 , wherein:
the original structure of the original source code class files is characterized by a first number of files or classes, and
the different structure of the compiled mutated bytecode class files is characterized by a second number of files or classes that is different from the first number.

15. The one or more non-transitory computer-readable media of claim 13 , wherein the computer-executable instructions cause the one or more processors to perform at least one of generating the mutated source code class files in parallel or compiling the mutated source code class files in parallel.

16. The one or more non-transitory computer-readable media of claim 13 , wherein:
the computer-executable instructions cause the one or more processors to:
determine code coverage metrics associated with a plurality of source class files by executing the test cases against an unmodified version of the software application; and
identify particular source class files, of the plurality of source code class files, that have, based on the code coverage metrics, at least a threshold code coverage level and correspond to the at least one mutation configuration, and

the original source code class files, modified to generate the mutated source code class files, comprise the particular source code class files.

17. A s

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Pearson Andrew L
• Shepherd Nate

Classifications

• G06F8/44

  Encoding · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06F11/3692

  for test results analysis · CPC title

• G06F8/30

  Creation or generation of source code · CPC title

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title