Selection of computer system deployments to mock for testing

Patent metadata

Field	Value
Publication number	US-11663112-B2
Application number	US-202017077742-A
Country	US
Kind code	B2
Filing date	Oct 22, 2020
Priority date	Oct 22, 2020
Publication date	May 30, 2023
Grant date	May 30, 2023

Abstract

Systems and methods facilitating selection of computer system deployments to mock for testing are described herein. A method as described herein can include filtering, by a first system operatively coupled to a processor, data relating to deployments of respective second systems as collected from the respective second systems, wherein the filtering is based on relevancy of the deployments to a testing criterion and results in a filtered group of deployments; grouping, by the first system, respective ones of the filtered group of deployments into respective clusters according to a difference function, the difference function being defined according to respective properties of the respective ones of the filtered group of deployments; and designating, by the first system, selected ones of the filtered group of deployments from respective ones of the clusters for testing according to a selection criterion.

First claim

What is claimed is:

1. A system, comprising:
a memory that stores executable components; and
a processor that executes the executable components stored in the memory, wherein the executable components comprise:
a filtering component that filters data relating to computing system deployments collected from respective computing systems based on relevance of the computing system deployments to a system testing criterion, resulting in filtered computing system deployments;
a clustering component that associates respective ones of the filtered computing system deployments with respective system clusters according to a difference function, the difference function being a distance function that computes intermediate distance measures corresponding to respective variable properties of configurations of the respective ones of the filtered computing system deployments;
a merit scoring component that assigns figures of merit to first ones of the configurations, of first ones of the filtered computing system deployments, based on an extent to which the first ones of the configurations are representative of second ones of the configurations, of second ones of the filtered computing system deployments, the second ones of the filtered computing system deployments being different from the first ones of the filtered computing system deployments and assigned to a same system cluster of the respective system clusters as the first ones of the filtered computing system deployments, as determined using a sum of distances between first intermediate distance measures, of the intermediate distance measures and associated with the first ones of the configurations, and second intermediate distance measures, of the intermediate distance measures and associated with the second ones of the configurations; and
a selection component that designates respective selected ones of the filtered computing system deployments for testing from respective ones of the respective system clusters based on the figures of merit assigned to the first ones of the configurations.

2. The system of claim 1 , wherein the system testing criterion comprises amounts of elapsed time from collection of the data relating to the computing system deployments.

3. The system of claim 1 , wherein the system testing criterion comprises a criterion based on usage by the computing system deployments of one or more software features designated for testing.

4. The system of claim 1 , wherein the respective variable properties are selected from the group consisting of client input/output activity volume associated with the filtered computing system deployments and configuration properties of software features utilized by the filtered computing system deployments.

5. The system of claim 4 , wherein the difference function is a function of a sum of a first difference measure, associated with the client input/output activity volume associated with the filtered computing system deployments, and a second difference measure, associated with the configuration properties of the software features utilized by the filtered computing system deployments.

6. The system of claim 1 , wherein the clustering component associates the filtered computing system deployments with the respective system clusters via hierarchical agglomerative clustering, and wherein the executable components further comprise:
a clustering control component that assigns a stopping criterion to the hierarchical agglomerative clustering performed by the clustering component.

7. The system of claim 6 , wherein the stopping criterion is selected from the group consisting of a target number of the respective system clusters and a threshold amount of intra-cluster variance between the configurations of the respective ones of the filtered computing system deployments as determined based on the difference function.

8. The system of claim 1 , wherein the merit scoring component assigns a first figure of merit, of the figures of merit, to a computing system deployment, of the filtered computing system deployments, based on historical data relating to malfunctions associated with the computing system deployment.

9. The system of claim 1 , wherein the selection component determines a number of the computing system deployments, of the filtered computing system deployments and associated with respective ones of the respective system clusters, to designate for testing in proportion to an amount of intra-cluster variance between the filtered computing system deployments.

10. The system of claim 1 , wherein the difference function comprises a weighted sum of feature difference measures for respective software features used by the filtered computing system deployments, and wherein respective ones of the feature difference measures are weighted by per-feature weights for the respective software features.

11. A method, comprising:
filtering, by a first system operatively coupled to a processor, data relating to deployments of second systems as collected from the second systems, wherein the filtering is based on relevancy of the deployments to a testing criterion and results in filtered deployments;
grouping, by the first system, respective ones of the filtered deployments into respective clusters according to a difference function, wherein the difference function satisfies properties of a distance function that computes intermediate distances corresponding to respective variable properties of configurations of the respective ones of the filtered deployments;
assigning, by the first system, respective figures of merit to first configurations, of the configurations and associated with first ones of the filtered deployments, based on an extent to which the first configurations are representative of second configurations, of the configurations and associated with second ones of the filtered deployments, as determined by a sum of distances between first intermediate distances, of the intermediate distances and associated with the first configurations, and second intermediate distances, of the intermediate distances and associated with the second configurations, wherein the second ones of the filtered deployments are not the first ones of the filtered deployments and are grouped into a same cluster of the respective clusters as the first ones of the filtered deployments; and
designating, by the first system, selected ones of the filtered deployments from respective ones of the clusters for testing based on the respective figures of merit assigned to the first configurations.

12. The method of claim 11 , wherein the testing criterion comprises a criterion selected from a group of criteria evaluated with respect to at least one of amounts of elapsed time from collection of the data relating to the deployments of the second systems or usage by the deployments of the second systems of one or more system features designated for testing.

13. The method of claim 11 , wherein the respective variable properties are selected from the group consisting of client input/output activity associated with the filtered deployments and configuration properties of software features utilized by the filtered deployments.

14. The method of claim 11 , wherein the grouping comprises grouping the filtered deployments into the respective clusters via hierarchical agglomerative clustering, and wherein the method further comprises:
causing, by the first system, the hierarchical agglomerative clustering to cease in response to satisfaction of a stopping condition.

15. The method of claim 14 , wherein the stopping condition comprises a threshold amount of intra-cluster variance being exh

Assignees

• Emc Ip Holding Co Llc

Inventors

• Bryan Matthew

Classifications

• G06F11/3698 Primary

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/301 Primary

  where the computing system is a virtual computing platform, e.g. logically partitioned systems (virtual machines G06F9/45533; logical partitioning of resources G06F9/5077) · CPC title

• G06F8/60

  Software deployment · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3072

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title