Methods and apparatus for systems determining a probable set of problems to explain symptoms

Patent metadata

Field	Value
Publication number	US-10176071-B1
Application number	US-201514674134-A
Country	US
Kind code	B1
Filing date	Mar 31, 2015
Priority date	Mar 31, 2015
Publication date	Jan 8, 2019
Grant date	Jan 8, 2019

Abstract

Methods and apparatus for performing event correlation using codebook processing including determining a most probable set of problems for observed symptoms in a system. In embodiments, a correlation matrix is received which has managed objects. Hypotheses are defined as a subset of problems having observed symptoms based on the correlation matrix and evaluated.

First claim

What is claimed is:

1. A method of performing event correlation using codebook processing including determining a most probable set of problems for observed symptoms in a system comprising an interconnected plurality of devices, each respective interconnected device providing respective data storage that is shareable with the other interconnected devices, the method comprising;
receiving, in real time, a correlation matrix for the system;
defining hypotheses as a subset of problems having observed symptoms in the system based on the correlation matrix, wherein there is a causal probability from problem set to symptoms;
determining a partial relative probability for each of the problems by parallel vertex processing including:
for each incoming edge for a problem vertex, determining the partial relative probability from the symptom and the problem for the problem vertex;
sending first messages that include the determined partial relative probability; and
combining first messages for the same problem vertex by multiplication;

partitioning the correlation matrix into a plurality of partitions according to the observed symptoms;
performing a best first search on each partition of the system, wherein determining an expansion of visited hypothesis nodes and their relative probabilities and upper bounds is calculated using parallel vertex processing including:
for outgoing edges for a problem vertex in the hypothesis, determining a sum of 1 and the additive inverse of the causal probability, sending second messages that include the determined sum, and combining second messages for the same vertex using multiplication;
for each outgoing edge for a symptom, determining a probability of that symptom's occurred value given the hypothesis, divided by the probability given the hypothesis unioned with a set containing the problem attached to an outgoing edge, divided by the probability given the problem attached to the outgoing edge, and determining the upper bound on those probabilities for the same;
sending third messages of the compute probabilities and upper bounds; and
combining third messages for the same problem vertex using multiplication;

for each problem receiving an incoming first, second or third message, combining the respective incoming first, second, or third message with the partial probability and upper bounds for that problem and the probability and upper bound for the hypothesis using multiplication to form the probability and upper bound for a child of the hypothesis
determining, based at least in part on the hypothesis and on the probability and upper bound for a child of the hypothesis, a real-time operational state of the system, the real-time operational state including information indicating, in real-time, the impacts of detected problems in the system; and
dynamically and automatically rebalancing the shareable data storage among the interconnected plurality of devices of the system, based at least in part on the real-time operational state, to compensate for the impacts of the detected problems.

2. The method according to claim 1 , wherein the partial relative probability for each of the problems is updated incrementally as the set of occurred symptoms change.

3. A method according to claim 1 , wherein the correlation matrix is partitioned according to the observed set of symptoms and calculation of the relative probability of a hypothesis in each partition is performed in parallel.

4. The method of claim 1 , wherein the system comprises a plurality of hosts interconnected over a network.

5. The method of claim 1 , wherein the determination of the probable set of problems occurs in real-time, and wherein each symptom comprises information relating to a real-time operational state of the system.

6. The method of claim 1 wherein the correlation matrix is created based at least in part on a set of dynamic, real time data, the set of dynamic, real-time data comprising topological data representing the plurality of hosts and a set of telemetry data from the plurality of hosts.

7. The method of claim 1 , further comprising monitoring one or more real-time states of the system based at least in part on one or more of the first messages, second messages, third messages, and the hypotheses.

8. A system for performing event correlation using codebook processing including determining a most probable set of problems for observed symptoms in a system comprising a plurality of interconnected devices, each respective interconnected device providing respective data storage that is shareable with the other interconnected devices, the system comprising:
a memory and a processor configured to:
receive, in real time, a correlation matrix for the system;
define hypotheses as a subset of problems having observed symptoms in the system based on the correlation matrix, wherein there is a causal probability from problem set to symptoms;
determine a partial relative probability for each of the problems by parallel vertex processing including:
for each incoming edge for a problem vertex, determine the partial relative probability from the symptom and the problem for the problem vertex;
send first messages that include the determined partial relative probability; and
combine first messages for the same problem vertex by multiplication;

partition the correlation matrix into a plurality of partitions according to the observed symptoms;
perform a best first search on each partition of the system, wherein determining an expansion of visited hypothesis nodes and their relative probabilities and upper bounds is calculated using parallel vertex processing including:
for outgoing edges for a problem vertex in the hypothesis, determine a sum of 1 and the additive inverse of the causal probability, sending second messages that include the determined sum, and combining second messages for the same vertex using multiplication;
for each outgoing edge for a symptom, determine a probability of that symptom's occurred value given the hypothesis, divided by the probability given the hypothesis unioned with a set containing the problem attached to an outgoing edge, divided by the probability given the problem attached to the outgoing edge, and determining the upper bound on those probabilities for the same;
send third messages of the compute probabilities and upper bounds; and
combine third messages for the same problem vertex using multiplication;

for each problem receiving an incoming first, second or third message, combine the respective incoming first, second, or third message with the partial probability and upper bounds for that problem and the probability and upper bound for the hypothesis using multiplication to form the probability and upper bound for a child of the hypothesis
determine, based at least in part on the hypothesis and on the probability and upper bound for a child of the hypothesis, a real-time operational state of the system, the real-time operational state including information indicating, in real-time, the impacts of detected problems in the system; and
dynamically and automatically rebalance the shareable data storage among the interconnected plurality of devices of the system, based at least in part on the real-time operational state, to compensate for the impacts of the detected problems.

9. The system according to claim 8 , wherein the partial relative probability for each of the problems is updated incrementally as the set of occurred symptoms change.

10. A system according to claim 8 , wherein the correlation matrix is partitioned according to the observed set of symptoms and calculation of the relative probability of a hypothesis in each partition is performed in parallel.

Assignees

• Emc Corp
• Emc Ip Holding Co Llc

Inventors

• Ohsie David
• Lam Cheuk

Classifications

• G06F11/3034 Primary

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F2201/805

  Real-time · CPC title

• G06F11/3447

  Performance evaluation by modeling · CPC title