Automating the production of runbook workflows

Patent metadata

Field	Value
Publication number	US-9891971-B1
Application number	US-82683310-A
Country	US
Kind code	B1
Filing date	Jun 30, 2010
Priority date	Jun 30, 2010
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

A method and program product for receiving an indication of a particular root cause associated with an error condition in an information technology (IT) system, electronically capturing activity of an operator for the IT system, automatically generating a workflow portion for a generic runbook, and storing the generic runbook in a manner that allows retrieval of the generic runbook.

First claim

What is claimed is:

1. A system comprising:
one or more processors;
a runbook repository enabled to contain a set of runbook automation workflows;
wherein a runbook automation flow is a series of non-human readable operational procedures executable by a computer to remedy the root cause of an error;

a runbook analysis module;
a root cause analysis module; and
computer executable logic enabled to be executed across the one or more processors of:
detecting an error by the root cause analysis module without user input;
in response to detection of the error, determining the root cause of the error without user input;
notifying the runbook analysis module of the error without user input;
determining, without user input, if a portion of a runbook automation flow in the runbook repository corresponds to the error;
based on a negative portion that no portion of a runbook corresponds to the root cause:
capturing user activity of a user correcting the error;
automatically generating, without user input via the computer system, a new runbook automation flow portion based on captured user activity; and
storing the new runbook automation flow portion, without user input via the computer system, in the runbook repository; wherein the new runbook automation flow portion is associated with the determined root cause of the error; and
enabling automatic retrieval and execution of the new runbook automation flow portion, without user interaction, based on identification of a second error with a second root cause similar to the first root cause.

2. The system of claim 1 wherein the logic if further enabled to:
based on a positive determination that a particular portion of a particular runbook automation flow in the runbook repository corresponds to the error, automatically retrieving and executing the particular portion of the particular runbook automation flow without user interaction.

3. The system of claim 2 wherein the logic is further enabled to:
detecting a second error;
determining a second root cause of the second error;
sending a notification of the root cause of the error to the runbook repository;
determining if any portion of any runbook automation flow exists is associated with the second root cause; and;
based on a determination that a particular portion of a particular runbook automation flow is associated with the second root cause, triggering without user interaction, the execution of the particular portion of the particular runbook automation flow.

4. The system of claim 2 wherein the logic is further enabled to:
determining if the particular portion of the runbook automation flow automatically corrected the error; and
based on a determination that the automatic execution of the particular runbook automation flow did not remedy the error;
determining a new root cause of the error;
automatically generating, via the computer system and without human interaction, a further runbook automation flow portion based on the captured activity; and
storing the further runbook automation flow portion, via the computer system, in the runbook repository; wherein the further runbook automation flow portion is associated with the new root cause of the error; and
enabling automatic retrieval and execution of the further runbook automation flow portion, without user interaction, based on identification of a third error with a root cause similar to the new root cause.

5. The system of claim 2 wherein the new runbook automation flow portion is an automatable format executable by a computer without human interaction.

6. The system of claim 1 wherein a workflow, generator uses a semantic model and bases the generating the new runbook automation flow at least partially on the semantic model.

7. The system of claim 1 wherein a workflow, generator uses a state machine model and bases the generating the new runbook automation flow at least partially on the state machine model wherein each procedure in the runbook automation flow represents a node in the state machine model.

8. A method comprising:
detecting, without user input, an error in an IT system;
in response to detection of the error and without user input, determining the root cause of the error;
notifying a runbook analysis module of the error;
determining if a portion of a runbook in a runbook repository corresponds to the error;
wherein the runbook repository is enabled to contain a set of runbook automation workflows; wherein a runbook automation flow is a series of non-human readable operational procedures executable by a computer to remedy the root cause of an error;
based on a negative portion that no portion of a runbook corresponds to the root cause:
capturing user activity of a user correcting the error;
automatically generating, without user input via the computer system, a new runbook automation flow portion based on captured user activity;
storing the new runbook automation flow portion, via the computer system, in the runbook repository; wherein the new runbook automation flow portion is associated with the determined root cause of the error; and enabling automatic retrieval and execution of the new runbook automation flow portion, without user interaction, based on identification of a second error with a second root cause similar to the first root cause.

9. The method of claim 8 further comprising:
based on a positive determination that a particular portion of a particular runbook automation flow in the runbook repository corresponds to the error, automatically retrieving and executing the particular portion of the particular runbook automation flow without user interaction.

10. The method of claim 9 further comprising:
detecting a second error;
determining a second root cause of the second error;
sending a notification of the root cause of the error to the runbook repository;
determining if any portion of any runbook automation flow exists is associated with the second root cause; and;
based on a determination that a particular portion of a particular runbook automation flow is associated with the second root cause, triggering without user interaction, the execution of the particular portion of the particular runbook automation flow.

11. The method of claim 9 further comprising:
determining if the particular portion of the runbook automation flow automatically corrected the error; and
based on a determination that the automatic execution of the particular runbook automation flow did not remedy the error;
determining a new root cause of the error;
automatically generating, via the computer system and without human interaction, a further runbook automation flow portion based on the captured activity;
storing the further runbook automation flow portion, via the computer system, in the runbook repository;
wherein the further runbook automation flow portion is associated with the new root cause of the error; and
enabling automatic retrieval and execution of the further runbook automation flow portion, without user interaction, based on identification of a third error with a root cause similar to the new root cause.

12. The method of claim 8 wherein the new runbook portion is an automatable format executable by a computer without human interaction.

13. The method of claim 8 wherein a workflow generator uses a semantic model and bases the generating the new runbook automation flow at least partially on the semantic model.

14. The system of claim 8 wherein a workflow generator uses a state machine model and bases the generating the new runbook automation flow at least partially on the state machine model w

Assignees

• Kuhhirte William F
• Emc Ip Holding Co Llc

Inventors

• Kuhhirte William F

Classifications

• G06F11/079 Primary

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title

• G06F11/0709 Primary

  in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems · CPC title