Detecting causes of performance regression to adjust data systems

Patent metadata

Field	Value
Publication number	US-10031831-B2
Application number	US-201514694022-A
Country	US
Kind code	B2
Filing date	Apr 23, 2015
Priority date	Apr 23, 2015
Publication date	Jul 24, 2018
Grant date	Jul 24, 2018

Abstract

At least one application in a computing environment is executed and one or more performance metrics of the application are measured. The measured performance metrics are analyzed and an operational performance regression is detected. The detected operational performance regression is correlated with one or more recorded changes and the correlated changes are identified as a cause of the operational performance regression. The elements of the computing environment are alerted in accordance with the identified changes to adjust operational performance.

First claim

What is claimed is:

1. A method of monitoring performance regression comprising:
recording one or more changes to a computing environment, wherein the computing environment includes at least one application, one or more jobs, and an operating system, and wherein recording one or more changes further comprises:
periodically identifying and recording the changes; and
determining one or more jobs impacted by the identified changes, wherein determining one or more jobs includes:
determining that an identified change impacts a plurality of different jobs based on the identified change being associated with a shared object;
identifying each job using the shared object as being impacted by the identified change; and
identifying each job of the computing environment as being impacted by the identified change in response to the identified change being associated with the operating system;

executing the at least one application in the computing environment and measuring one or more performance metrics;
analyzing the measured performance metrics and detecting an operational performance regression;
correlating the detected operational performance regression with one or more recorded changes, wherein correlating the detected operational performance regression further comprises:
determining one or more jobs associated with the detected operational performance regression; and
obtaining the changes impacting the determined jobs to correlate the changes with the detected performance regression;

identifying a correlated change as a cause of the operational performance regression; and
adjusting execution of the determined jobs and operational performance of the computing environment to remedy the detected performance regression by altering steps of the determined jobs causing the detected performance regression indicated by the identified changes.

2. The method of claim 1 , wherein the performance metrics include one or more from a group of: elapsed time, CPU usage, memory usage, disk input/output usage, and network input/output usage.

3. The method of claim 1 , further comprising:
executing the at least one application in the computing environment to produce an initial baseline of performance metrics.

4. The method of claim 3 , wherein the operational performance regression is detected based on a measured performance metric deviating in excess of a corresponding standard deviation from the initial baseline.

5. The method of claim 1 , wherein the one or more changes include changes to one or more from a group of: elements of the computing environment, the at least one application, and the operating system.

6. A system for monitoring performance regression in a computing environment, the system comprising:
a processor configured to:
record one or more changes to the computing environment, wherein the computing environment includes at least one application, one or more jobs, and an operating system, and wherein recording one or more changes further comprises:
periodically identifying and recording the changes; and
determining one or more jobs impacted by the identified changes, wherein determining one or more jobs includes:
determining that an identified change impacts a plurality of different jobs based on the identified change being associated with a shared object;
identifying each job using the shared object as being impacted by the identified change; and
identifying each job of the computing environment as being impacted by the identified change in response to the identified change being associated with the operating system;

execute the at least one application in the computing environment and measure one or more performance metrics;
analyze the measured performance metrics and detect an operational performance regression;
correlate the detected operational performance regression with one or more recorded changes, wherein correlating the detected operational performance regression further comprises:
determining one or more jobs associated with the detected operational performance regression; and
obtaining the changes impacting the determined jobs to correlate the changes with the detected performance regression;

identify a correlated change as a cause of the operational performance regression; and
adjust execution of the determined jobs and operational performance of the computing environment to remedy the detected performance regression by altering steps of the determined jobs causing the detected performance regression indicated by the identified changes.

7. The system of claim 6 , wherein the performance metrics include one or more from a group of: elapsed time, CPU usage, memory usage, disk input/output usage, and network input/output usage.

8. The system of claim 6 , wherein the processor is further configured to:
execute the at least one application in the computing environment to produce an initial baseline of performance metrics.

9. The system of claim 8 , wherein the operational performance regression is detected based on a measured performance metric deviating in excess of a corresponding standard deviation from the initial baseline.

10. The system of claim 6 , wherein the one or more changes include changes to one or more from a group of: elements of the computing environment, the at least one application, and the operating system.

11. A computer program product for monitoring performance regression in a computer environment, comprising a non-transitory computer-readable storage medium having computer-readable program code embodied thereon, the computer-readable program code, when executed by a processor, causes the processor to:
record one or more changes to the computing environment, wherein the computing environment includes at least one application, one or more jobs, and an operating system, and wherein recording one or more changes further comprises:
periodically identifying and recording the changes; and
determining one or more jobs impacted by the identified changes, wherein determining one or more jobs includes:
determining that an identified change impacts a plurality of different jobs based on the identified change being associated with a shared object;
identifying each job using the shared object as being impacted by the identified change; and
identifying each job of the computing environment as being impacted by the identified change in response to the identified change being associated with the operating system;

execute the at least one application in the computing environment and measure one or more performance metrics;
analyze the measured performance metrics and detect an operational performance regression;
correlate the detected operational performance regression with one or more recorded changes, wherein correlating the detected operational performance regression further comprises:
determining one or more jobs associated with the detected operational performance regression; and
obtaining the changes impacting the determined jobs to correlate the changes with the detected performance regression;

identify a correlated change as a cause of the operational performance regression; and
adjust execution of the determined jobs and operational performance of the computing environment to remedy the detected performance regression by altering steps of the determined jobs causing the detected performance regression indicated by the identified changes.

12. The computer program product of claim 11 , the computer-readable program code being further configured to cause the processor to:
execute the at least one application in the computing environment to produce an initial baseline of performance metrics, wherein th

Assignees

• IBM

Inventors

• Greene Lawrence A
• Jacobson Eric A
• Li Yong
• Pu Xiaoyan

Classifications

• G06F2201/865

  Monitoring of software · CPC title

• G06F11/3466

  Performance evaluation by tracing or monitoring · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title

• G06F11/3612 Primary

  by runtime analysis (performance monitoring G06F11/3466) · CPC title