Self-learning automated remediation of changes that cause performance degradation of applications

Patent metadata

Field	Value
Publication number	US-9876673-B2
Application number	US-201414315169-A
Country	US
Kind code	B2
Filing date	Jun 25, 2014
Priority date	Jun 25, 2014
Publication date	Jan 23, 2018
Grant date	Jan 23, 2018

Abstract

Techniques are disclosed for automatic remediation of application performance degradations caused by configuration changes. In one embodiment, a learning module keeps track of application configuration changes and subsequent effects on the application's performance. The learning module creates new potential remediation rules based on correlations between such configuration changes and performance degradations or improvements. The learning module affirms such potential rules if the correlation between the configuration changes and degradations or improvements are repeatedly observed, and vice versa. When subsequent performance degradations are observed, a rule engine, which maintains a set of remediation rules, evaluates the rules to identify configuration changes relevant to the observed performance degradation and determines whether the probability that the configuration changes caused the degradation are greater than a threshold for invoking a remediation action, such as rolling back the configuration changes.

First claim

We claim:

1. A method for remedying performance degradations of an application deployed in a computer system, said method comprising:
learning one or more rules applicable to remedy performance degradations of the application, wherein the learning of each of the rules includes:
identifying a correlation between an observed configuration change and an observed degradation or improvement in performance of the application,
creating, based on the identified correlation, a potential rule not applied to remedy performance degradations, wherein the potential rule specifies at least the application, configuration change criteria determined based on the observed configuration change, and at least one of degradation criteria or a remediation action determined based on the observed degradation or improvement, respectively, and
promoting the potential rule to a rule actively applied to remedy performance degradations in response to a ranking of the potential rule satisfying a threshold value based on repeated observations of the correlation between the configuration change and the performance degradation or improvement;

receiving notification of a first performance degradation;
identifying, based on configuration change criteria and degradation criteria in the learned rules, one or more configuration changes to the application relevant to the first performance degradation;
determining, based on one or more of the learned rules, whether each of the identified configuration changes satisfies a corresponding probability threshold indicating that the configuration change caused the first performance degradation; and
reconfiguring the application based on one or more remediation actions specified in one or more of the learned rules if one or more of the identified configuration changes satisfies the corresponding probability thresholds.

2. The method of claim 1 ,
wherein a ranking of a given rule or potential rule is increased if a correlation between a configuration change and a degradation or improvement in performance associated with the given rule or potential rule is repeatedly observed,
wherein the ranking of the given rule or potential rule is decreased if the correlation between the configuration change and the degradation or improvement in performance associated with the given rule or potential rule is not repeatedly observed,
wherein the given potential rule is promoted to a rule actively used to remedy performance degradations if the ranking of the given potential rule exceeds a first threshold value, and
wherein the given rule is removed from a set of rules actively used to remedy performance degradations if the ranking of the given rule falls below a second threshold value.

3. The method of claim 1 , wherein learning the rules further includes verifying that applying the rules when associated performance degradations are observed results in expected performance improvements.

4. The method of claim 1 , wherein the observations of the correlation between the configuration change and the degradation or improvement in performance are aggregated observations for multiple instances of the application.

5. The method of claim 1 , wherein the observed configuration change is a user configuration change that undoes a previous configuration change.

6. The method of claim 1 , wherein reconfiguring the application includes rolling back the one or more identified configuration changes when the corresponding probability thresholds are satisfied.

7. The method of claim 1 , wherein each of the learned rules includes configuration change criteria and degradation criteria that need to be met for the rule to apply, a formula for determining probability that one or more degradations are caused by one or more configuration changes, and a probability threshold for taking remediation action to reconfigure the application.

8. A non-transitory computer readable storage medium comprising instructions to carry out a method for remedying performance degradations of an application deployed in a computer system, said method comprising:
learning one or more rules applicable to remedy performance degradations of the application, wherein the learning of each of the rules includes:
identifying a correlation between an observed configuration change and an observed degradation or improvement in performance of the application,
creating, based on the identified correlation, a potential rule not applied to remedy performance degradations, wherein the potential rule specifies at least the application, configuration change criteria determined based on the observed configuration change, and at least one of degradation criteria or a remediation action determined based on the observed degradation or improvement, respectively, and
promoting the potential rule to a rule actively applied to remedy performance degradations in response to a ranking of the potential rule satisfying a threshold value based on repeated observations of the correlation between the configuration change and the performance degradation or improvement;

receiving notification of a first performance degradation;
identifying, based on configuration change criteria and degradation criteria in the learned rules, one or more configuration changes to the application relevant to the first performance degradation;
determining, based on one or more of the learned rules, whether each of the identified configuration changes satisfies a corresponding probability threshold indicating that the configuration change caused the first performance degradation; and
reconfiguring the application based on one or more remediation actions specified in one or more of the learned rules if one or more of the identified configuration changes satisfies the corresponding probability thresholds.

9. The non-transitory computer readable storage medium of claim 8 ,
wherein a ranking of a given rule or potential rule is increased if a correlation between a configuration change and a degradation or improvement in performance associated with the given rule or potential rule is repeatedly observed,
wherein the ranking of the given rule or potential rule is decreased if the correlation between the configuration change and the degradation or improvement in performance associated with the given rule or potential rule is not repeatedly observed,
wherein the given potential rule is promoted to a rule actively used to remedy performance degradations if the ranking of the given potential rule exceeds a first threshold value, and
wherein the given rule is removed from a set of rules actively used to remedy performance degradations if the ranking of the given rule falls below a second threshold value.

10. The non-transitory computer readable storage medium of claim 8 , wherein learning the rules further includes verifying that applying the rules when associated performance degradations are observed results in expected performance improvements.

11. The non-transitory computer readable storage medium of claim 8 , wherein the observations of the correlation between the configuration change and the degradation or improvement in performance of the application are aggregated observations for multiple instances of the application.

12. The non-transitory computer readable storage medium of claim 8 , wherein the observed configuration change is a user configuration change that undoes a previous configuration change.

13. The non-transitory computer readable storage medium of claim 8 , wherein reconfiguring the application includes rolling back the one or more identified configuration changes when the corresponding probability thresholds are satisfied.

Assignees

• Vmware Inc

Inventors

• Margalit Adar
• Dvir Eran

Classifications

• H04L41/0631 Primary

  using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis · CPC title

• H04L41/0672

  Electricity · mapped topic

• H04L41/0863

  by rolling back to previous configuration versions · CPC title

• H04L41/0661

  by reconfiguring faulty entities · CPC title