Parallel software testing based on annotations

Patent metadata

Field	Value
Publication number	US-2025199945-A1
Application number	US-202519069722-A
Country	US
Kind code	A1
Filing date	Mar 4, 2025
Priority date	Feb 23, 2021
Publication date	Jun 19, 2025
Grant date	—

Abstract

Annotations associated with test cases written to test a software application can allow related test cases to be dynamically distributed among different sets of test cases that can be executed simultaneously in different parallel threads, thereby speeding up testing relative to executing the test cases sequentially in a single thread. The annotations can also allow test cases to be identified that are relevant to code changes, such that testing times can be further reduced by executing a subset of test cases relevant to code changes instead of a full set of test cases.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
identifying, by a computing system comprising a processor, a plurality of annotated test cases associated with a software application; determining, by the computing system, a first annotation that corresponds to a code change; selecting, by the computing system, and from among the plurality of annotated test cases based on the first annotation, a set of test cases associated with the code change; determining, by the computing system, second annotations that differ from the first annotation and respectively indicate corresponding test topics; dividing, by the computing system, the set of test cases into different subsets of related test cases that respectively correspond with different annotations of the second annotations; and executing, by the computing system, two or more of the different subsets of related test cases in parallel.

2 . The computer-implemented method of claim 1 , wherein the code change comprises a change to at least one test case of the plurality of annotated test cases.

3 . The computer-implemented method of claim 1 , wherein the code change comprises a change to source code of the software application.

4 . The computer-implemented method of claim 1 , wherein the software application is configured to manage policies based on database records stored in a database.

5 . The computer-implemented method of claim 4 , wherein at least one of the test topics, corresponding to the second annotations, is associated with testing one or more types of data interactions between the software application and the database.

6 . The computer-implemented method of claim 4 , wherein at least one of the test topics, corresponding to the second annotations, is associated with testing lifecycle operations associated with management of one or more stages of a lifecycle of a policy.

7 . The computer-implemented method of claim 4 , wherein at least one of the test topics, corresponding to the second annotations, is associated with testing integration operations associated with integration of the software application with one or more external systems.

8 . The computer-implemented method of claim 1 , further comprising:
determining, by the computing system, and using a machine learning model, annotations associated with the plurality of annotated test cases, wherein the machine learning model is trained on training data that indicates example test topics associated with example test cases.

9 . The computer-implemented method of claim 1 , further comprising:
causing, by the computing system, display of an annotations tool in a user interface of a programming environment associated with the plurality of annotated test cases, wherein the annotations tool is configured to:
present a list of selectable annotations;
receive a user selection of a particular annotation; and
associate the particular annotation with at least one test case of the plurality of annotated test cases.

10 . The computer-implemented method of claim 1 , further comprising:
collecting, by the computing system, a plurality of test results generated based on execution of the different subsets of related test cases; and aggregating, by the computing system, the plurality of test results.

11 . A computing system, comprising:
one or more processors; and memory storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to:
identify a plurality of annotated test cases associated with a software application;
determine a first annotation that corresponds to a code change;
select, from among the plurality of annotated test cases based on the first annotation, a set of test cases associated with the code change;
determine second annotations that differ from the first annotation and respectively indicate corresponding test topics;
divide the set of test cases into different subsets of related test cases that respectively correspond with different annotations of the second annotations; and
execute two or more of the different subsets of related test cases in parallel.

12 . The computing system of claim 11 , wherein the code change comprises a change to at least one of:
a test case of the plurality of annotated test cases, or source code of the software application.

13 . The computing system of claim 11 , wherein the software application is configured to manage policies based on database records stored in a database.

14 . The computing system of claim 13 , wherein at least one of the test topics, corresponding to the second annotations, is associated with testing at least one of:
data interactions between the software application and the database, lifecycle operations associated with management of one or more stages of a lifecycle of a policy, or integration operations associated with integration of the software application with one or more external systems.

15 . The computing system of claim 11 , wherein the computer-executable instructions further cause the computing system to determine annotations associated with the plurality of annotated test cases based on at least one of:
predictions of the annotations generated by a machine learning model trained on training data that indicates example test topics associated with example test cases, or user selections of the annotations via an annotations tool presented in a user interface of a programming environment associated with the plurality of annotated test cases.

16 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors of a computing system, cause the computing system to:
identify a plurality of annotated test cases associated with a software application; determine a first annotation that corresponds to a code change; select, from among the plurality of annotated test cases based on the first annotation, a set of test cases associated with the code change; determine second annotations that differ from the first annotation and respectively indicate corresponding test topics; divide the set of test cases into different subsets of related test cases that respectively correspond with different annotations of the second annotations; and execute two or more of the different subsets of related test cases in parallel.

17 . The one or more non-transitory computer-readable media of claim 16 , wherein the code change comprises a change to at least one of:
a test case of the plurality of annotated test cases, or source code of the software application.

18 . The one or more non-transitory computer-readable media of claim 16 , wherein the software application is configured to manage policies based on database records stored in a database.

19 . The one or more non-transitory computer-readable media of claim 18 , wherein at least one of the test topics, corresponding to the second annotations, is associated with testing at least one of:
data interactions between the software application and the database, lifecycle operations associated with management of one or more stages of a lifecycle of a policy, or integration operations associated with integration of the software application with one or more external systems.

20 . The one or more non-transitory computer-readable media of claim 16 , wherein the computer-executable instructions further cause the computing system to determine annotations associated with the plurality of anno

Assignees

• State Farm Mutual Automobile Insurance Co

Inventors

• Chauhan Shaktiraj
• Shepherd Nate

Classifications

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F8/73

  Program documentation · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G06F11/3692

  for test results analysis · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title