Phased parameterized combinatoric testing for a data storage system

Patent metadata

Field	Value
Publication number	US-11842782-B2
Application number	US-202117491080-A
Country	US
Kind code	B2
Filing date	Sep 30, 2021
Priority date	Sep 30, 2021
Publication date	Dec 12, 2023
Grant date	Dec 12, 2023

Abstract

Phased parameterized combinatoric testing for a data storage system is disclosed. A testing recipe can be performed according to different input arguments. Combinatoric testing of the data storage system can be based on different combinations of operations and arguments. The disclosed testing can employ a consistent integer index for arguments passed into the sequenced operations of the recipe. The recipe can be employed to generate a phased test tree that can enable testing based on a phase rather than loading an entire test suite into memory. The consistent integer index can be used to identify failed test cases such that the entire test can be reconstituted from stored failed test information. Distribution of test cases to worker process can based on the phased test tree to facilitate interning an operation. Stored failed test information can include human-readable failure information.

First claim

What is claimed is:

1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining test recipe data representative of a test recipe for combinatoric testing of a data storage system;
determining topology data representative of a phased testing tree topology associated with a phased test tree based on the test recipe data;
distributing test cases to worker processes based on the phased test tree, wherein the distributing of the test cases to the worker processes is performed for a phase of the phased testing tree topology loaded into the memory, and wherein the phased testing tree topology is not loaded entirely into the memory; and
storing failure information corresponding to failed test cases.

2. The system of claim 1 , wherein the test recipe comprises a plurality of operations.

3. The system of claim 2 , wherein an operation of the plurality of operations is associated with a first argument and a second argument, wherein the first and second arguments are different arguments and wherein, when the operation is performed in accord with the first argument, a first result occurs that is different from a second result corresponding to performing the operation in accord with the second argument.

4. The system of claim 1 , wherein the phased testing tree topology is divisible into phases separated by a synchronization point.

5. The system of claim 4 , wherein the synchronization point corresponds to a serial operation that is to be performed on all surviving test cases subsequent to completion of a preceding phase of the phased testing tree topology.

6. The system of claim 1 , wherein the data storage system is not a distributed data storage system, and wherein the worker processes are located local to the processor of the system.

7. The system of claim 1 , wherein the data storage system is a distributed data storage system, and wherein the worker processes are located local to the processor of the system.

8. The system of claim 1 , wherein the data storage system is a distributed data storage system, and wherein a worker process of the worker processes is located remotely from the processor of the system.

9. The system of claim 1 , wherein the test cases are identified via respective unique identifiers corresponding to combinatoric parameters of the operations comprised in the test recipe.

10. The system of claim 1 , wherein the storing of the failure information comprises storing human-readable failure information.

11. The system of claim 1 , wherein the storing of the failure information facilitates regenerating all test cases resulting from the test recipe.

12. The system of claim 1 , wherein the storing of the failure information enables regenerating a failed test case without traversing the phased test tree from a root node to a failed node corresponding to the failed test case.

13. The system of claim 1 , wherein the distributing of the test cases to the worker processes comprises determining a distribution that maintains parameterized testing nodes of a branch of the phased testing tree topology at a same worker process to enable interning of an operation of the test recipe at the same worker process.

14. The system of claim 1 , wherein testing test cases in accord with the phased test tree results in a failed test case of the failed test cases, and wherein the phased test tree is pruned in response to the failed test case.

15. A method, comprising:
receiving, by a system comprising a processor, a test recipe for combinatoric testing of a data storage system;
determining, by the system, a topology of a phased test tree based on the test recipe, wherein the topology is divisible into phases separated by a serial operation comprised in the test recipe;
distributing, by the system, test cases to worker processes in accord with the phased test tree;
coordinating, by the system, performance of test cases represented in each phase of the topology without loading all phases of the topology concurrently into memory; and
storing, by the system, failure information corresponding to failed test cases resulting from iteratively undergoing performance in each phase of the phased test tree.

16. The method of claim 15 , wherein the data storage system is a distributed data storage system, and wherein the distributing of the test cases comprises distributing the test cases to the worker processes distributed among physical nodes of the distributed data storage system.

17. The method of claim 15 , wherein the coordinating of the performance of the test cases comprises performing an interned instance of an operation, wherein the interned instance is selected based on a unique identifier of a test case of the test cases, and wherein the unique identifier corresponds to combinatoric parameter values of operations comprised in the test recipe.

18. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
determining a test recipe for combinatoric testing of a distributed data storage system;
determining a phased testing tree topology corresponding to a phased test tree based on the test recipe;
distributing test cases to distributed worker processes of the distributed data storage system in accord with the phased test tree;
coordinating performance of test cases represented in individual phases of the phased testing tree topology without loading the phased test tree topology entirely into memory; and
storing failure information corresponding to failed test cases resulting from iteratively performing each phase of the phased test tree.

19. The non-transitory machine-readable medium of claim 18 , wherein a serial operation comprised in the test recipe is treated as a synchronization point of the test recipe, and wherein the serial operation corresponds to a division between different phases of the phased testing tree topology.

20. The non-transitory machine-readable medium of claim 18 , wherein the test recipe comprises a plurality of operations.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Bryan Matthew

Classifications

• G11C29/10 Primary

  Test algorithms, e.g. memory scan [MScan] algorithms; Test patterns, e.g. checkerboard patterns · CPC title

• G06F9/4881

  Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues · CPC title

• G06F9/522

  Barrier synchronisation · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title