Two-tier capacity planning

Patent metadata

Field	Value
Publication number	US-11995381-B2
Application number	US-201916455455-A
Country	US
Kind code	B2
Filing date	Jun 27, 2019
Priority date	Apr 26, 2019
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Computing devices, computer-readable storage media, and computer-implemented methods are disclosed for prediction of capacity. In a central tier, central-tier benchmark values are generated from benchmark testing performed on different test configurations in a reference execution environment. In a deployment tier, deployment-tier benchmark values are generated from benchmark testing performed on a baseline deployed configuration in many execution environments. A sizing model is learned from the central-tier benchmark values to predict execution platform requirements given a set of workload input parameters. A performance model is learned from the deployment-tier and the central-tier benchmark values to predict a performance delta value reflecting relative performance between a particular execution environment and the reference execution environment. The performance delta value is used to adjust predicted execution platform requirements to tailor the prediction to a particular execution environment. The predicted execution platform requirements can be deployed and tested to validate or tune the performance model.

First claim

What is claimed is:

1. A computer-implemented method comprising: generating a performance delta value representing a predicted performance difference between hardware in a particular execution environment and reference hardware in a reference execution environment, by processing an encoded representation of a plurality of benchmark values representing observed performance measurements of the particular execution environment in a trained performance model;
generating one or more execution platform requirements that size the reference execution environment to handle one or more workload input parameters by processing an encoded representation of the one or more workload input parameters in a trained sizing model;
generating one or more adjusted execution platform requirements for the particular execution environment by adjusting the one or more execution platform requirements for the reference execution environment with the performance delta value; and
executing an operation in the particular execution environment, wherein the particular execution environment is configured using the one or more adjusted execution platform requirements.

2. The computer-implemented method of claim 1 , the trained sizing model trained based on first benchmark testing on a plurality of test configurations deployed on the reference hardware in the reference execution environment, the one or more benchmark values generated based on second benchmark testing on a baseline deployed configuration deployed on the hardware of the particular execution environment distinct from the reference execution environment, wherein the performance delta value reflects a difference between performance by the reference hardware and the hardware of the particular execution environment.

3. The computer-implemented method of claim 1 , the trained sizing model trained based on first benchmark testing on a plurality of test configurations deployed in the reference execution environment in a central tier, the trained performance model trained based on second benchmark testing on a plurality of deployed configurations deployed in a deployment tier comprising a plurality of execution environments distinct from the reference execution environment.

4. The co mputer-implemented method of claim 1 , the trained performance model trained based on benchmark testing on a plurality of deployed configurations comprising a baseline deployed configuration deployed in each of a plurality of execution environments distinct from the reference execution environment.

5. The co mputer-implemented method of claim 1 , the trained performance model trained based on benchmark testing on a baseline deployed configuration comprising a single instance of an application operating as a search head and an indexer.

6. The computer-implemented method of claim 1 , the trained performance model trained based on benchmark testing performed by a benchmark test application provided by an application-provider that operates the reference execution environment.

7. The computer-implemented method of claim 1 , the one or more benchmark values collected by a benchmark test application configured to trigger a simulation of a defined performance test workload and collect the one or more benchmark values based on the simulation.

8. The computer-implemented method of claim 1 , the one or more benchmark values representing indexing performance value and search performance of the particular execution environment.

9. The computer-implemented method of claim 1 , the one or more benchmark values collected by a benchmark test application configured to trigger a simulation and adapt at least one simulation value of the simulation to coincide with one or more break points of the particular execution environment.

10. The computer-implemented method of claim 1 , wherein executing the operation comprises:
prompting an option to perform benchmark testing on a deployed configuration implementing the one or more adjusted execution platform requirements in the particular execution environment;
receiving, from the particular execution environment, one or more subsequent benchmark values representing one or more subsequent observed performance measures of the particular execution environment generated by the benchmark testing; and
using the one or more sub sequent benchmark values to tune the trained performance model.

11. The computer-implemented method of claim 1 , the one or more observed performance measures comprising at least one of indexing throughput or a search count observed during benchmark testing of the particular execution environment.

12. The computer-implemented method of claim 1 , wherein generating the performance delta value is further based on applying a representation of hardware specifications for the particular execution environment and the representation of the one or more benchmark values to the trained performance model.

13. One or more computer-readable storage media having instructions stored thereon, wherein the instructions, when executed by a computing device, cause the computing device to perform operations comprising:
generating a performance delta value representing a predicted performance difference between hardware in a particular execution environment and reference hardware in a reference execution environment, by processing an encoded representation of a plurality of benchmark values representing observed performance measurements of the particular execution environment in a trained performance model;
generating one or more execution platform requirements that size the reference execution environment to handle one or more workload input parameters, by processing an encoded representation of the one or more workload input parameters in a trained sizing model;
generating one or more adjusted execution platform requirements for the particular execution environment by adjusting the one or more execution platform requirements for the reference execution environment with the performance delta value; and
executing an operation in the particular execution environment, wherein the particular execution environment is configured using the one or more adjusted execution platform requirements.

14. The one or more computer-readable storage media of claim 13 , the trained sizing model trained based on first benchmark testing on a plurality of test configurations deployed on the reference hardware in the reference execution environment, the one or more benchmark values generated based on second benchmark testing on a baseline deployed configuration deployed on the hardware of the particular execution environment distinct from the reference execution environment, wherein the performance delta value reflects a difference between performance by the reference hardware and the hardware of the particular execution environment.

15. The one or more computer-readable storage media of claim 13 , the trained sizing model trained based on first benchmark testing on a plurality of test configurations deployed in the reference execution environment in a central tier, the trained performance model trained based on second benchmark testing on a plurality of deployed configurations deployed in a deployment tier comprising a plurality of execution environments distinct from the reference execution environment.

16. The one or more computer-readable storage media of claim 13 , the trained performance model trained based on benchmark testing on a plurality of deployed configurations comprising a baseline deployed configuration deployed in each of a plurality of execution environments distinct from the reference execu

Assignees

• Splunk Inc

Inventors

• Cai Jie
• Cao Yang
• He Ning
• Pei Bing
• Ye Xiaolu
• Yu Chong
• Zhang Aiping
• Zhou Zhou

Classifications

• G06F30/20 Primary

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06F8/77 Primary

  Software metrics · CPC title

• G06F9/455

  Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines · CPC title

• G06F11/3428

  Benchmarking · CPC title

• G06F11/3447

  Performance evaluation by modeling · CPC title