Methods and systems for executing tests using grouped/filtered test classes during testing of an application

Patent metadata

Field	Value
Publication number	US-2017075791-A1
Application number	US-201615140803-A
Country	US
Kind code	A1
Filing date	Apr 28, 2016
Priority date	Sep 14, 2015
Publication date	Mar 16, 2017
Grant date	—

Abstract

Computer implemented methods and systems are provided for executing tests in a system that includes a user system and a cloud-based computing platform communicatively coupled to the user system. The user system includes a processing system, memory and an input system that receives input parameters specified by a user of the user system. The memory can store a test class filter module executable by the processing system. Upon being executed by the processing system, the test class filter module can, based on one or more of the input parameters, group and filter test class identifiers to generate a unique test class identifier array of filtered test class identifiers that correspond to a particular subset of tests that are to be executed during testing of an application. A test execution engine of the cloud-based computing platform can then execute the particular subset of tests corresponding to the filtered test class identifiers specified in the unique test class identifier array.

First claim

What is claimed:

1 . A system, comprising:
a user system, comprising:
an input system configured to receive input parameters specified by a user of the user system;
a processing system; and
memory configured to store: a test class filter module executable by the processing system, wherein the test class filter module, upon being executed by the processing system, is configured to: group and filter test class identifiers, based on one or more of the input parameters, to generate a unique test class identifier array of filtered test class identifiers that correspond to a particular subset of tests that are to be executed during testing of an application; and
a cloud-based computing platform, communicatively coupled to the user system, and comprising:
a test execution engine configured to execute the particular subset of tests corresponding to the filtered test class identifiers specified in the unique test class identifier array.

2 . The system according to claim 1 , wherein the cloud-based computing platform further comprises: an interface that interfaces with the test class filter module, wherein the input parameters comprise: a manifest file specified by the user of the user system, and wherein the test class filter module comprises:
a first test class fetcher comprising:
a manifest file processor configured to: retrieve the manifest file specified by the user of the user system, wherein the manifest file comprises a plurality of test class names that can be used to fetch a particular group of test class identifiers from a data store of the cloud-based computing platform; and
a manifest file reader and query builder configured to: read the test class names from the manifest file, send the test class names from the manifest file to the interface, receive the test class identifiers corresponding to each of the test class names from the manifest file from the interface, and group the test class identifiers into a first test class identifier array.

3 . The system according to claim 2 , wherein the manifest file has a name, and wherein the plurality of test class names in the manifest file comprises:
a statically defined set of test class names each having a corresponding test class identifier that identifies a test class to be executed.

4 . The system according to claim 3 , wherein the application is a multi-tenant application, and wherein the user selects the manifest file to specify the statically defined set of test class names that are of interest to a particular sub-tenant of the multi-tenant application during testing of the multi-tenant application.

5 . The system according to claim 1 , wherein the input parameters comprise:
one or more regular expressions specified by the user of the user system, wherein each regular expression is a pattern that is used to perform a pattern match to identify a particular group of one or more test class identifiers associated with that regular expression that each correspond to a test class name of a test class, wherein the test class filter module further comprises:
a second test class fetcher comprising:
a regular expression processor configured to receive the regular expressions input by the user of the user system; and
a query builder configured to: generate and send a query associated with each regular expression to the interface,
wherein the interface is configured to receive each query, contact the data store and find matching test class names for each regular expression, and return a test class identifier for each of the matching test class names to the query builder, and
wherein the query builder is further configured to receive the test class identifiers for each of the matching test class names, and group the test class identifiers corresponding to each of the regular expressions into a second test class identifier array.

6 . The system according to claim 5 , wherein the application is a multi-tenant application, and wherein the user selects the one or more regular expressions to specify the particular group of one or more test class names that are of interest to a particular sub-tenant of the multi-tenant application during testing of the multi-tenant application.

7 . The system according to claim 5 , wherein the test class filter module further comprises:
a unique class ID filter configured to: receive the first test class identifier array and the second test class identifier array; and filter the test class identifiers from the first test class identifier array and the test class identifiers from the second test class identifier array to generate the unique test class identifier array of filtered test class identifiers by:
combining the test class identifiers from the first test class identifier array and the test class identifiers from the second test class identifier array into a combined set of test class identifiers;
determining duplicate test class identifiers that are in the combined set;
determining invalid test class identifiers that are in the combined set; and
eliminating the duplicate test class identifiers and the invalid test class identifiers from the combined set to generate the unique test class identifier array of filtered test class identifiers that correspond to the particular subset of tests that are to be executed.

8 . The system according to claim 7 , wherein the unique test class identifier array of filtered test class identifiers correspond to a particular subset of all possible test classes that have been defined by the user via the input parameters.

9 . The system according to claim 8 , wherein the application is a multi-tenant application, wherein the user is a sub-tenant, and wherein the particular subset of all possible test classes are those that are of interest to that particular sub-tenant and correspond to the particular subset of tests that are to be executed during testing of the multi-tenant application.

10 . The system according to claim 7 , wherein the memory is further configured to store a test management module comprising:
a test enqueuer module executable by the processing system, wherein the test enqueuer module, upon being executed by the processing system, is configured to:
receive the unique test class identifier array of filtered test class identifiers from the test class filter module; and
communicate the filtered test class identifiers as a batch to the test execution engine, wherein the test execution engine is configured to execute the particular subset of tests corresponding to each of the filtered test class identifiers specified in the unique test class identifier array; and
a test result poller/fetcher executable by the processing system, wherein the test result poller/fetcher, upon being executed by the processing system, is configured to:
receive, from the test execution engine, a parent job identifier used to track test execution status of the particular subset of tests corresponding to each of the filtered test class identifiers that are to be executed at the test execution engine;
use the parent job identifier to poll the test execution engine to monitor the progress of the particular subset of tests; and
when the test result poller/fetcher determines that all tests in the particular subset of tests have been executed by the test execution engine:
communicate the parent job identifier to the interface to request the test results and trigger retrieval of the test results from the data store.

11 . The system according to claim 10 , wherein the input parameters comprise:
a test execution time limit specified by the user of the user system, wherein the test enqueuer module is configured to receive the test execution time limi

Assignees

• Salesforce Com Inc

Inventors

• Ramakrishna Adarsh
• Glaser Scott
• Rao Gandham Vamshidhar
• Vengareddy Ashwin Simha Reddy
• Morales Jonathan Rico

Classifications

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3684

  for test design, e.g. generating new test cases · CPC title

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G06F11/3664 Primary

  Physics · mapped topic

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title