Automated testing of code modules against individually relevant data sets

Patent metadata

Field	Value
Publication number	US-2018113798-A1
Application number	US-201615332080-A
Country	US
Kind code	A1
Filing date	Oct 24, 2016
Priority date	Oct 24, 2016
Publication date	Apr 26, 2018
Grant date	—

Abstract

A testing server tests new code modules, or re-tests previously tested code modules, using individualized sets of test data. The testing server receives test datasets from an execution engine, which runs tested code modules to process operational datasets of computing devices. The testing server receives an untested code module from a user, and obtains test dataset parameters applicable to the untested code module. The testing server generates an individualized set of test datasets based on the test dataset parameters. The testing server tests the untested code module by processing each test dataset in the individualized set of test datasets to produce test results.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
receiving a plurality of test datasets from an execution engine that runs a plurality of tested code modules to process operational datasets of at least one computing device; receiving an untested code module from a user client; obtaining test dataset parameters applicable to the untested code module; generating a first set of test datasets from the plurality of test datasets based on the test dataset parameters; and testing the untested code module by processing each test dataset in the first set of test datasets to produce test results.

2 . The method of claim 1 , further comprising notifying an author of the untested code module with the test results.

3 . The method of claim 1 , wherein the test dataset parameters include one or more of a first number of most recently received test datasets, a second number of worst-case test datasets, or a third number of tagged test datasets.

4 . The method of claim 3 , further comprising determining whether each particular test dataset of the plurality of test datasets is a worst-case test dataset based on results of the particular test dataset being processed by the plurality of tested code modules on the execution engine.

5 . The method of claim 3 , wherein the tagged test datasets are derived from the operational datasets of the at least one computing device.

6 . The method of claim 1 , further comprising:
re-testing a candidate code module of the tested code modules as the untested code module; and based on unsatisfactory results of the re-testing, removing the candidate code module from the plurality of tested code modules.

7 . The method of claim 6 , wherein each of the tested code modules is periodically re-tested as the untested code module.

8 . An apparatus comprising:
a network interface unit configured to communicate with an execution engine that runs a plurality of tested code modules to process operational datasets of at least one computing device; a memory; and a processor coupled to the network interface unit and memory, the processor configured to:
receive a plurality of test datasets from the execution engine via the network interface unit;
receive an untested code module from a user client;
obtain test dataset parameters applicable to the untested code module;
generate a first set of test datasets from the plurality of test datasets based on the test dataset parameters; and
test the untested code module by processing each test dataset in the first set of test datasets to produce test results.

9 . The apparatus of claim 8 , wherein the processor is further configured to notify an author of the untested code module with the test results.

10 . The apparatus of claim 8 , wherein the test dataset parameters include one or more of a first number of most recently received test datasets, a second number of worst-case test datasets, or a third number of tagged test datasets.

11 . The apparatus of claim 10 , wherein the processor is further configured to determine whether each particular test dataset of the plurality of test datasets is a worst-case test dataset based on results of the particular test dataset being processed by the plurality of tested code modules on the execution engine.

12 . The apparatus of claim 10 , wherein the tagged test datasets are derived from the operational datasets of the at least one computing device, the tagged test datasets including one or more tags associated with the at least one computing device, wherein the one or more tags include a Virtual Private Network (VPN) tag, a Network Address Translation (NAT) tag, a Session Initiation Protocol (SIP) tag, a routing tag, or a security tag.

13 . The apparatus of claim 8 , wherein the processor is further configured to:
re-test a candidate code module of the tested code modules as the untested code module; and based on unsatisfactory results of the re-testing, remove the candidate code module from the plurality of tested code modules.

14 . The apparatus of claim 13 , wherein the processor is further configured to periodically re-test each of the tested code modules as the untested code module.

15 . One or more non-transitory computer readable storage media encoded with computer executable instructions operable to cause a processor to:
receive a plurality of test datasets from an execution engine that runs a plurality of tested code modules to process operational datasets of at least one computing device; receive an untested code module from a user client; obtain test dataset parameters applicable to the untested code module; generate a first set of test datasets from the plurality of test datasets based on the test dataset parameters; and test the untested code module by processing each test dataset in the first set of test datasets to produce test results.

16 . The computer readable storage media of claim 15 , further comprising instructions operable to cause the processor to notify an author of the untested code module with the test results.

17 . The computer readable storage media of claim 15 , wherein the test dataset parameters include one or more of a first number of most recently received test datasets, a second number of worst-case test datasets, or a third number of tagged test datasets.

18 . The computer readable storage media of claim 17 , further comprising instructions operable to cause the processor to determine whether each particular test dataset of the plurality of test datasets is a worst-case test dataset based on results of the particular test dataset being processed by the plurality of tested code modules on the execution engine.

19 . The computer readable storage media of claim 17 , wherein the tagged test datasets are derived from the operational datasets of the at least one computing device, the tagged test datasets including one or more tags associated with the at least one computing device, wherein the one or more tags include a Virtual Private Network (VPN) tag, a Network Address Translation (NAT) tag, a Session Initiation Protocol (SIP) tag, a routing tag, or a security tag.

20 . The computer readable storage media of claim 15 , further comprising instructions operable to cause the processor to:
periodically re-test a candidate code module of the tested code modules as the untested code module; and based on unsatisfactory results of the re-testing, removing the candidate code module from the plurality of tested code modules.

Assignees

• Cisco Tech Inc

Inventors

• Johnston Jay K
• Mortensen Magnus
• White Jr David C

Classifications

• G06F11/3672 Primary

  Test management · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3684 Primary

  for test design, e.g. generating new test cases · CPC title

• G06F11/3612

  by runtime analysis (performance monitoring G06F11/3466) · CPC title