Method and apparatus for generating simulated test IO operations

Patent metadata

Field	Value
Publication number	US-11748241-B2
Application number	US-202117151776-A
Country	US
Kind code	B2
Filing date	Jan 19, 2021
Priority date	Jan 19, 2021
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

Different storage platforms and different applications generate different types of traffic. Separate learning processes are trained using live IO operations from different types of reference workloads. The reference workloads are based on multiple types of applications executing in multiple types of environments. Example applications may include Oracle, SAP, SQL, etc. Example environments may include finance/banking, retail, on-line transaction processing, healthcare, etc. Once the learning processes have been trained, trained model checkpoints describing the trained learning processes are stored in a reference workload repository. The trained model checkpoints are used in a testing environment to enable a test server to create a test model configured to generate simulated test IO operations to be applied to a storage system. The generated test IO operations simulate the workload from an intended application and environment where the storage system will be used.

First claim

What is claimed is:

1. A method of generating simulated Input/Output (IO) operations of an application, comprising:
training a learning process using IO operations of a reference workload generated by an executing instance of the application on a first storage system as training examples, to cause the learning process to learn workload characteristics of the reference workload of the executing instance of the application, the workload characteristics of the reference workload include a number of logical devices on which the IO operations were generated, a range of sizes of the IO operations of the reference workload, a sequentiality of the IO operations of the reference workload, a ratio of read vs write IO operations, and a range of addresses of the IO operations of the reference workload;
transmitting trained model checkpoints describing the learned workload characteristics of the reference workload from the first storage system to a reference workload repository;
storing the trained model checkpoints in the reference workload repository;
determining that a new instance of the application is to be deployed that, once deployed, will generate IO operations associated with a new workload on a second storage system;
retrieving the trained model checkpoints from the reference workload repository, by a test server;
using the trained model checkpoints to configure a test model in the test server;
determining an overall volume of simulated IO operations per second (IOPS) to be generated by the test model, the overall volume of IOPS to be generated being based on a determined overall volume of IO operations expected to be generated as the new workload by the new instance of the application once the new instance of the application is deployed;
using the test model, by the test server, to synthetically generate the overall volume of IOPS of simulated application IO operations, the generated simulated application IO operations having a number of logical devices, a range of size of IO operations, a sequentiality of IO operations, a ratio of read vs write IO operations, and a range of addresses, that are all based on the learned workload characteristics of the reference workload; and
applying the overall volume of IO operations per second of simulated application IO operations, by the test server, to the second storage system to test a response of the second storage system to a simulated reference workload of the new instance of the application, prior to deploying the new instance of the application.

2. The method of claim 1 , further comprising training multiple learning processes based on different reference workloads generated by different applications executing in different environments; and
storing trained model checkpoints describing each respective trained learning process in the reference workload repository.

3. The method of claim 2 , further comprising storing application metadata with respective trained model checkpoints in the reference workload repository, the application metadata identifying the application and environment that generated the reference workload that was used to train the respective learning process.

4. The method of claim 1 , wherein the learning process is a Long Short-Term Memory Neural Network or a Recurrent Neural Network.

5. The method of claim 1 , further comprising creating the number of logical devices used by the application on the second storage system, and populating the devices with data prior to generating the synthetically generated IO operations and prior to applying the synthetically generated IO operations to the second storage system.

6. A non-transitory tangible computer readable storage medium having stored thereon a computer program for generating simulated Input/Output (IO) operations of an application, the computer program including a set of instructions which, when executed by a computer, cause the computer to perform a method comprising the steps of:
training a learning process using IO operations of a reference workload generated by an executing instance of the application on a first storage system as training examples, to cause the learning process to learn workload characteristics of the reference workload of the executing instance of the application, the workload characteristics of the reference workload include a number of logical devices on which the IO operations were generated, a range of sizes of the IO operations of the reference workload, a sequentiality of the IO operations of the reference workload, a ratio of read vs write IO operations, and a range of addresses of the IO operations of the reference workload;
transmitting trained model checkpoints describing the learned workload characteristics of the reference workload from the first storage system to a reference workload repository;
storing the trained model checkpoints in the reference workload repository;
determining that a new instance of the application is to be deployed that, once deployed, will generate IO operations associated with a new workload on a second storage system;
retrieving the trained model checkpoints from the reference workload repository, by a test server;
using the trained model checkpoints to configure a test model in the test server;
determining an overall volume of simulated IO operations per second (IOPS) to be generated by the test model, the overall volume of IOPS to be generated being based on a determined overall volume of IO operations expected to be generated as the new workload by the new instance of the application once the new instance of the application is deployed;
using the test model, by the test server, to synthetically generate the overall volume of IOPS of simulated application IO operations, the generated simulated application IO operations having a number of logical devices, a range of size of IO operations, a sequentiality of IO operations, a ratio of read vs write IO operations, and a range of addresses, that are all based on the learned workload characteristics of the reference workload; and
applying the overall volume of IO operations per second of simulated application IO operations, by the test server, to the second storage system to test a response of the second storage system to a simulated reference workload of the new instance of the application, prior to deploying the new instance of the application.

7. The non-transitory tangible computer readable storage medium of claim 6 , further comprising training multiple learning processes based on different reference workloads generated by different applications executing in different environments; and
storing trained model checkpoints describing each respective trained learning process in the reference workload repository.

8. The non-transitory tangible computer readable storage medium of claim 7 , further comprising storing application metadata with respective trained model checkpoints in the reference workload repository, the application metadata identifying the application and environment that generated the reference workload that was used to train the respective learning process.

9. The non-transitory tangible computer readable storage medium of claim 6 , wherein the learning process is a Long Short-Term Memory Neural Network or a Recurrent Neural Network.

10. The non-transitory tangible computer readable storage medium of claim 6 , further comprising creating the number of logical devices used by the application on the second storage system, and populating the devices with data prior to generating the synthetically generated IO operations and prior to applying the synthetically generated IO operations to the second storage system.

11. A system for generating simulated Input/Ou

Assignees

• Emc Ip Holding Co Llc
• Dell Products Lp

Inventors

• Martin Owen
• Raghavan Rasa

Classifications

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06F11/3688 Primary

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/1407

  Checkpointing the instruction stream · CPC title

• G06F11/3414 Primary

  Workload generation, e.g. scripts, playback · CPC title