Predictive diagnosis of SLA violations in cloud services by seasonal trending and forecasting with thread intensity analytics

Patent metadata

Field	Value
Publication number	US-9495395-B2
Application number	US-201314109578-A
Country	US
Kind code	B2
Filing date	Dec 17, 2013
Priority date	Apr 11, 2013
Publication date	Nov 15, 2016
Grant date	Nov 15, 2016

Abstract

Data can be categorized into facts, information, hypothesis, and directives. Activities that generate certain categories of data based on other categories of data through the application of knowledge which can be categorized into classifications, assessments, resolutions, and enactments. Activities can be driven by a Classification-Assessment-Resolution-Enactment (CARE) control engine. The CARE control and these categorizations can be used to enhance a multitude of systems, for example diagnostic system, such as through historical record keeping, machine learning, and automation. Such a diagnostic system can include a system that forecasts computing system failures based on the application of knowledge to system vital signs such as thread or stack segment intensity and memory heap usage. These vital signs are facts that can be classified to produce information such as memory leaks, convoy effects, or other problems. Classification can involve the automatic generation of classes, states, observations, predictions, norms, objectives, and the processing of sample intervals having irregular durations.

First claim

What is claimed is:

1. A computer-implemented method comprising:
obtaining a series of thread dump samples by generating a separate thread dump during each particular time interval in a series of time intervals;
generating a set of stack segment classifications based on the series of thread dump samples, wherein generating the set of stack segment classifications comprises;
identifying a first set of stack frames within a first stack segment each of the first set of stack frames having a corresponding intensity value;
determining that the corresponding intensities of the first set of stack frames within the first stack segment diverge over time beyond a specified threshold;
splitting the first stack segment into multiple constituent stack segments, based on the determination that the corresponding intensities of the first set of stack frames within the first stack segment diverge over time beyond the specified threshold; and
generating a new stack segment classification for each of the multiple constituent stack segments;

for each particular thread represented in the series of thread dump samples, selecting, from the set of stack segment classifications, a particular stack segment classification for the particular thread based at least in part on a number of invocations of a code block in the particular stack segment multiplied by an execution time of the code block;
determining a trend for each stack segment classification in the set of stack segment classifications, thereby generating a set of trends;
determining a set of anomalies based at least in part on the set of trends; and
generating output based at least in part on the set of anomalies;
wherein said set of anomalies includes at least one of: a level spike, a level shift, a level drift, a variance change, a saturation, a stuck thread, a lingering thread, a deadlock condition, a congestion upstream, a congestion downstream, a congestion in resource pool, a convoy effect, an impedance mismatch, an outlying measurement, or a burst of outlying measurements.

2. The method of claim 1 , further comprising:
determining whether a thread name, an Execution Context ID (ECID), or an epoch counter of a thread represented in a first thread dump is the same as, respectively, a thread name, an Execution Context ID (ECID), or an epoch counter of a thread represented in a second thread dump;
in response to determining that the thread name, Execution Context ID (ECID), or epoch counter of the thread represented in the first thread dump is the same as, respectively, the thread name, Execution Context ID (ECID), or epoch counter of the thread represented in the second thread dump, preventing the thread represented in the second thread dump from being counted for intensity statistics.

3. The method of claim 2 , further comprising:
determining that a CPU time of the thread represented in the first thread dump is same as a CPU time of a thread represented in the second thread dump; and in response to determining that the thread name, Execution Context ID (ECID), or epoch counter of the thread represented in the first thread dump is the same as, respectively, the thread name, Execution Context ID (ECID), or epoch counter of the thread represented in the second thread dump and that the CPU time of the thread represented in the first thread dump is same as the CPU time of a thread represented in the second thread dump, identifying the same thread represented in the first and second thread dump as a stuck thread.

4. The method of claim 1 , further comprising:
determining, based at least in part on a lack of change in a plurality of CPU times of a first thread within separate thread dump samples in the series of thread dump samples, that the first thread is stuck.

5. The method of claim 1 , further comprising:
determining that a CPU time of a thread represented in a first thread dump is different from a CPU time of the thread represented in a second thread dump; and
in response to determining that the CPU time of the thread represented in the first thread dump is different from the CPU time of a thread represented in the second thread dump, identifying the same thread represented in the first and second thread dump as a lingering thread.

6. The method of claim 5 , further comprising:
determining, based at least in part on a lack of change in a plurality of the thread name, Execution Context ID (ECID), or epoch counter of a first thread within separate thread dump samples in the series of thread dump samples, that a first thread is lingering.

7. The method of claim 1 , further comprising:
maintaining a first intensity statistics of a first sampling interval for thread dumps in the series of thread dumps;
adjusting a second intensity statistics of a second sampling interval for thread dumps in the series of thread dumps in response to a determination that a particular thread is determined to be stuck or lingering over a first and second sampling intervals, wherein a second sampling interval avoids duplicate counting of a stuck or lingering thread.

8. The method of claim 1 , wherein selecting, from the set of stack segment classifications, a particular stack segment classification for the particular thread comprises:
determining an intensity value for a particular stack frame in a particular stack trace;
determining an intensity value for a predecessor stack frame that occurs immediately before the particular stack frame in the particular stack trace;
determining an intensity value for a successor stack frame that occurs immediately after the particular stack frame in the particular stack trace;
in response to determining that the intensity value for the particular stack frame is within a predetermined threshold of the intensity value for the predecessor stack frame, assigning a particular classification to both the particular stack frame and the predecessor stack frame; and
in response to determining that the intensity value for the particular stack frame is the same as the intensity value for the successor stack frame, assigning the particular classification to both the particular stack frame and the successor stack frame.

9. The method of claim 1 , further comprising:
creating a new stack segment information item;
assigning the new stack segment information item to a value of a current stack segment information item;
generating a new seasonal trend information item in response to a classification of the new stack segment information item; and
setting a value of a trend attribute of the current stack segment information item to be the new seasonal trend information item.

10. The method of claim 1 , further comprising:
splitting a coalesced stack segment into a first new stack segment and a second new stack segment;
creating a first clone of a seasonal trend information item of the coalesced stack segment;
creating a second clone of the seasonal trend information item of the coalesced stack segment;
setting a value of a trend attribute of the first new stack segment to be the first clone; and
setting a value of a trend attribute of the second new stack segment to be the second clone.

11. The method of claim 1 , further comprising:
generating a set of thread classifications based at least in part on a chain of stack segment classifications; and
generating a classification of a chain of threads based at least in part on one of an invocation chain, a dependency chain, and a client-server chain among threads.

12. The method of claim 1 , further comprising:
organizing the set of stack segment classifications by assigning one or more stack segment classifications to a dimension;
setting a name of the

Assignees

• Oracle Int Corp

Inventors

• Chan Eric S
• Ahad Rafiul
• Ghoneimy Adel
• Santos Adriano Covello

Classifications

• G06F11/0751

  Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06) · CPC title

• G06F16/1824

  implemented using Network-attached Storage [NAS] architecture (distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097) · CPC title

• G06F16/285

  Clustering or classification · CPC title

• G06F11/3471

  Address tracing · CPC title

• G06F11/0718

  in an object-oriented system · CPC title