Mutation testing within continuous integration systems

Patent metadata

Field	Value
Publication number	US-2024419580-A1
Application number	US-202418814901-A
Country	US
Kind code	A1
Filing date	Aug 26, 2024
Priority date	Apr 7, 2021
Publication date	Dec 19, 2024
Grant date	—

Abstract

Techniques described herein relate to implementing mutation testing of software applications associated with continuous integration (CI) systems. A mutation test system may determine one or more portions of modified source code within an application codebase. Mutated applications may be generated based on the modified source code, and a mutation test system may determine subsets application test suites for execution based on the portions of the modified source code and/or other factors. In various examples, the mutation test system may use mappings between portions of source code and test subsets, and/or machine-learned models or heuristics-based techniques to determine subsets of test suites based on discreet source code modifications. Mutation testing can be performed by executing the determined test subsets on the mutated applications, and the results may be used by the CI system to control the integration of the code changes into the shared source code repository and/or automated testing of the application build.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
receiving, by a computer system, a request to integrate a source code change into a codebase associated with an application; determining a line of source code associated with the requested source code change; determining, by the computer system, a source code attribute of the line of source code; determining an application test suite associated with the application; determining a test subset of the application test suite, based on the source code attribute of the line of source code; mutating, by the computer system, the source code change into a mutated source code change; building, by the computer system, a mutated application by compiling one or more software classes including the mutated source code change; and executing, by the computer system, the test subset of the application test suite on the mutated application.

2 . The computer-implemented method of claim 1 , further comprising:
determining that the source code change modifies a second line of source code within the codebase; determining a second test subset of the application test suite associated with the second line of source code, wherein the second test subset is different from the test subset; and executing the second test subset on the mutated application.

3 . The computer-implemented method of claim 1 , further comprising:
initiating an operation to integrate the source code change into the codebase, based at least in part on test results associated with executing the test subset.

4 . The computer-implemented method of claim 1 , wherein determining the test subset comprises:
inputting the source code attribute of the line of source code into a trained machine learning model configured to output data identifying the test subset of the application test suite.

5 . The computer-implemented method of claim 1 , wherein determining the test subset comprises:
retrieving test results associated with a previous execution of the application test suite, on a previous mutated application associated with the line of source code; and determining, within the test results associated with the previous execution, one or more passing tests; and excluding the one or more passing tests from the test subset.

6 . The computer-implemented method of claim 1 , wherein determining the test subset comprises:
accessing a mapping storing associations between a plurality of subsets of the application test suite, and associated source code attributes.

7 . The computer-implemented method of claim 1 , wherein:
mutating the source code change is based on a set of code mutation rules; and determining the test subset is based at least in part on the set of code mutation rules.

8 . The computer-implemented method of claim 1 , further comprising:
receiving a mutation test confidence level associated with the request to integrate the source code change; and determining the test subset based at least in part on the mutation test confidence level.

9 . A computer system, comprising:
one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a request to integrate a source code change into a codebase associated with an application;
determining a line of source code associated with the requested source code change;
determining a source code attribute of the line of source code;
determining an application test suite associated with the application;
determining a test subset of the application test suite, based on the source code attribute of the line of source code;
mutating the source code change into a mutated source code change;
building a mutated application by compiling one or more software classes including the mutated source code change; and
executing the test subset of the application test suite on the mutated application.

10 . The computer system of claim 9 , the operations further comprising:
determining that the source code change modifies a second line of source code within the codebase; determining a second test subset of the application test suite associated with the second line of source code, wherein the second test subset is different from the test subset; and executing the second test subset on the mutated application.

11 . The computer system of claim 9 , the operations further comprising:
initiating an operation to integrate the source code change into the codebase, based at least in part on test results associated with executing the test subset.

12 . The computer system of claim 9 , wherein determining the test subset comprises:
inputting the source code attribute of the line of source code into a trained machine learning model configured to output data identifying the test subset of the application test suite.

13 . The computer system of claim 9 , wherein determining the test subset comprises:
retrieving test results associated with a previous execution of the application test suite, on a previous mutated application associated with the line of source code; and determining, within the test results associated with the previous execution, one or more passing tests; and excluding the one or more passing tests from the test subset.

14 . The computer system of claim 9 , wherein determining the test subset comprises:
accessing a mapping storing associations between a plurality of subsets of the application test suite, and associated source code attributes.

15 . The computer system of claim 9 , wherein:
mutating the source code change is based on a set of code mutation rules; and determining the test subset is based at least in part on the set of code mutation rules.

16 . The computer system of claim 9 , the operations further comprising:
receiving a mutation test confidence level associated with the request to integrate the source code change; and determining the test subset based at least in part on the mutation test confidence level.

17 . One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed by the processor, cause the processor to perform operations comprising:
receiving a request to integrate a source code change into a codebase associated with an application; determining a line of source code associated with the requested source code change; determining a source code attribute of the line of source code; determining an application test suite associated with the application; determining a test subset of the application test suite, based on the source code attribute of the line of source code; mutating the source code change into a mutated source code change; building a mutated application by compiling one or more software classes including the mutated source code change; and executing the test subset of the application test suite on the mutated application.

18 . The one or more non-transitory computer-readable media of claim 17 ,
wherein determining the test subset comprises: inputting the source code attribute of the line of source code into a trained machine learning model configured to output data identifying the test subset of the application test suite.

19 . The one or more non-transitory computer-readable media of claim 17 , wherein determining the test subset comprises:
accessing a mapping storing associations between a

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Pearson Andrew L
• Shepherd Nate

Classifications

• G06F11/3692

  for test results analysis · CPC title

• G06F11/3676

  for coverage analysis · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/368 Primary

  for test version control, e.g. updating test cases to a new software version · CPC title