Mutation testing in parallel threads

Patent metadata

Field	Value
Publication number	US-2025068549-A1
Application number	US-202418948219-A
Country	US
Kind code	A1
Filing date	Nov 14, 2024
Priority date	Apr 7, 2021
Publication date	Feb 27, 2025
Grant date	—

Abstract

Mutation testing can indicate whether mutants of a software application, created by intentionally altering source code of the software application, are successfully “killed” by test cases executed against the mutants. Mutation testing can be performed via parallel threads by, within each parallel thread, modifying individual source code class files and recompiling the modified class files to generate and test mutants. Individual mutation test results produced within each of the parallel threads can be aggregated to generate an aggregated test result report that indicates overall testing metrics associated with the mutation testing across the parallel threads.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
executing, by a computing system comprising a processor, test cases against an unmodified version of a software application; determining, by the computing system, and based on execution of the test cases, code coverage metrics corresponding to a plurality of source code files associated with the software application; identifying, by the computing system, and based on the code coverage metrics, mutatable files, of the plurality of source code files, that:
correspond to at least one mutation configuration, and
have at least a threshold code coverage level;
generating, by the computing system, mutated source code files by modifying the mutatable files based on the at least one mutation configuration; compiling, by the computing system, the mutated source code files into bytecode files; and executing, by the computing system, and in parallel, the test cases against application mutants that include one or more of the bytecode files.

2 . The computer-implemented method of claim 1 , wherein:
the bytecode files have a different structure than an original structure of the mutated source code files, the original structure of the mutated source code files is characterized by a first number of files, and the different structure of the bytecode files is characterized by a second number of files that is different from the first number of files.

3 . The computer-implemented method of claim 1 , wherein:
the bytecode files have a different structure than an original structure of the mutated source code files, the original structure of the mutated source code files is characterized by a first number of classes, and the different structure of the bytecode files is characterized by a second number of classes that is different from the first number of classes.

4 . The computer-implemented method of claim 1 , wherein instances of at least one of generating the mutated source code files or compiling the mutated source code files are performed in parallel.

5 . The computer-implemented method of claim 1 , wherein executing the test cases against the application mutants comprises executing, in parallel, the test cases against different application mutants, the different application mutants comprising different sets of the bytecode files that are characterized by different modifications to the mutatable files.

6 . The computer-implemented method of claim 1 , wherein executing the test cases against the application mutants comprises executing different sets of test cases, in parallel, against a particular application mutant of the application mutants.

7 . The computer-implemented method of claim 1 , further comprising generating the application mutants by swapping, in memory, first previously-compiled bytecode files with the one or more of the bytecode files that correspond to the mutated source code files, without recompiling additional second previously-compiled bytecode files.

8 . The computer-implemented method of claim 1 , further comprising:
logging, by the computing system, mutation test results associated with executing the test cases against the application mutants in parallel; and aggregating, by the computing system, the mutation test results into an aggregated test result report.

9 . A computing system, comprising:
one or more processors; and memory storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to:
identify mutatable files, of a plurality of source code files associated with a software application, that:
correspond to at least one mutation configuration, and
have at least a threshold percentage of source code lines covered by a set of test cases;

generate mutated source code files by modifying the mutatable files based on the at least one mutation configuration;
compile the mutated source code files into bytecode files; and
execute, in parallel, test cases of the set of test cases against application mutants comprising one or more of the bytecode files.

10 . The computing system of claim 9 , wherein the bytecode files have a different structure than an original structure of the mutated source code files.

11 . The computing system of claim 10 , wherein:
the original structure of the mutated source code files is characterized by a first number of files or classes, and the different structure of the bytecode files is characterized by a second number of files or classes that is different from the first number.

12 . The computing system of claim 9 , wherein:
a first mutated source code file, of the mutated source code files, comprises a single mutated class, and the bytecode files comprise two or more classes that correspond to the single mutated class.

13 . The computing system of claim 9 , wherein the computer-executable instructions cause the computing system to perform at least one of generating the mutated source code files or compiling the mutated source code files in parallel.

14 . The computing system of claim 9 , wherein the computer-executable instructions cause the computing system to generate the application mutants by swapping, in the memory or a second memory, first previously-compiled bytecode files with the one or more of the bytecode files corresponding to the mutated source code files, without recompiling additional second previously-compiled bytecode files.

15 . The computing system of claim 14 , wherein different application mutants are generated by swapping different first previously-compiled bytecode files with different bytecode files.

16 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to:
execute a set of test cases against an unmodified version of a software application; determine, based on executing the set of test cases, code coverage metrics corresponding to a plurality of source code files associated with the software application; identify, based on the code coverage metrics, mutatable files, of the plurality of source code files, that:
correspond to at least one mutation configuration, and
have at least a threshold code coverage level;
generate mutated source code files by modifying the mutatable files based on the at least one mutation configuration; compile the mutated source code files into mutated bytecode files; and perform mutation testing by executing, in parallel, test cases of the set of test cases against different application mutants that comprise different sets of the mutated bytecode files.

17 . The one or more non-transitory computer-readable media of claim 16 , wherein the mutated bytecode files have a different structure than an original structure of the mutated source code files.

18 . The one or more non-transitory computer-readable media of claim 17 , wherein:
the original structure of the mutated source code files is characterized by a first number of files or classes, and the different structure of the mutated bytecode files is characterized by a second number of files or classes that is different from the first number.

19 . The one or more non-transitory computer-readable media of claim 16 , wherein the computer-executable instructions cause the one or more processors to perform at least one of generating the mutated source code files or compiling the mutated source code files in parallel.

20 . The one or more non-transitory comp

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Pearson Andrew L
• Shepherd Nate

Classifications

• G06F8/44

  Encoding · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06F11/3692

  for test results analysis · CPC title

• G06F8/30

  Creation or generation of source code · CPC title

• G06F11/368

  for test version control, e.g. updating test cases to a new software version · CPC title