Method and system for software system performance diagnosis with kernel event feature guidance

Patent metadata

Field	Value
Publication number	US-9075912-B2
Application number	US-201313850562-A
Country	US
Kind code	B2
Filing date	Mar 26, 2013
Priority date	Mar 26, 2012
Publication date	Jul 7, 2015
Grant date	Jul 7, 2015

Abstract

A method includes generating a normal trace in a training stage for the monitored software systems and a monitored trace in the deployment stage for anomaly detection, applying resource transfer functions to traces to convert them to resource features, and system call categorization to traces to convert them to program behavior features, performing anomaly detection in a global scope using the derived resource features and program behavior features, in case the system finds no anomaly, generating no anomaly report, in case the anomaly is found, including the result in an anomaly report; and performing conditional anomaly detection.

First claim

The invention claimed is:

1. A method for software system performance diagnosis with kernel event feature guidance, comprising the steps of:
receiving input information for low level kernel events from a training execution of the software system and a production system monitoring for an anomaly;
processing the input information in a trace generation providing a first set of transaction traces from a normal training scenario and a second set of traces generated from the same system in the production scenario;
determining by an anomaly trace localization procedure whether there exists an anomaly by comparing information from the first and second set of traces in a global scope;
applying a fine grained anomaly check by an anomaly event localization procedure responsive to an anomaly determined by the preceding anomaly trace localization procedure; and
obtaining an anomaly status from the determining or applying steps.

2. The method of claim 1 , wherein said anomaly trace localization procedure includes generating system resource features comprising: generating a new perspective of trace information in various resource usages, said trace information being system resource features, defining resource transfer functions for converting kernel events to a set of resource information in a uniform way; mapping between trace events and resource information possibly not being one-to-one; measuring network activity by a number of kernel events related to networking; and information responsive to the measuring step being encoded into a data structure called a system resource vector.

3. The method of claim 1 , wherein said anomaly trace localization procedure includes generating program behavior features comprises:
providing a perspective of trace information showing what application code triggered kernel event; and
differentiating traces caused by different codes.

4. The method of claim 1 , wherein said anomaly trace localization includes trace clustering comprising:
clustering traces by using program behavior features, said clustering using connectivity based clustering with a threshold in a distance function with a bottom-up approach and single-linkage being used to connect clusters.

5. The method of claim 1 , wherein said anomaly trace localization includes behavior feature matching comprising:
determining which clusters are comparable responsive to the first and second sets of input traces being clusters,
comparing traces from the first and second sets of input traces for clusters in a matched group;
representing traces generated by application code patterns that are different from a training run for unmatched clusters, the unmatched clusters being analyzed in an aggregated way of by way of manual specification; and
basing matching criteria of behavior features on a combination of system call distance and common kinds of system calls.

6. The method of claim 1 , wherein said anomaly trace localization includes conditional data mining comprising;
detecting an anomaly in trace clusters that belong to a match set using predetermined probability factors;
detecting an anomaly in unmatched clusters responsive to traces which do not have matched program behavior features indicating that their application logic is altered from ones in a training stage, statistics of these traces showing how much portion of whole traces become different in a monitored execution; and
performing a finer grained analysis in an event level on clusters for which any anomaly is found in a comparison of a matched cluster pair.

7. The method of claim 1 , wherein said anomaly event localization comprises:
a normal trace policy said normal trace policy can be based on choosing a trace having a closest resource value to a predetermined average value and this policy or it can be based on choosing a trace that has a smallest distance in program behavior features from an anomaly trace;
an anomaly trace policy for choosing a trace with a resource value with a biggest difference from said predetermined average value;
a comparison trace policy for comparing traces; and
an event filter for filtering out in a preprocessing stage random events which occur non-deterministically, such as interrupts, and can be in traces with kernel events representing application characteristics.

8. The method of claim 7 , wherein said comparison trace policy comprises choosing a transaction unit having a closest system call distance to another transaction unit, with compares traces possibly not having a same number of transaction units, pairs of transaction units can be flexibly matched and after that a selected pair of transaction units is compared using a longest common subsequence procedure with differences being manually examined to understand a root cause of the anomaly.

9. A system for software system performance diagnosis with kernel event feature guidance, comprising:
a training execution of the software system and a production system monitoring for an anomaly for kernel events inputs;
trace executions from training and monitored executions for providing a first set of transaction traces from a normal training scenario and a second set of traces generated from the same system in the production scenario;
an anomaly trace localization procedure for determining whether there exists an anomaly by comparing information from the first and second set of traces in a global scope; and
an anomaly event localization procedure for applying a fine grained anomaly check responsive to an anomaly determined by the preceding anomaly trace localization procedure.

10. The system of claim 9 , wherein said anomaly trace localization procedure includes generating system resource features comprising:
generating a new perspective of trace information in various resource usages, said trace information being system resource features,
defining resource transfer functions for converting kernel events to a set of resource information in a uniform way;
mapping between trace events and resource information possibly not being one-to-one;
measuring network activity by a number of kernel events related to networking; and
information responsive to the measuring step being encoded into a data structure called a system resource vector.

11. The system of claim 9 , wherein said anomaly trace localization procedure includes generating program behavior features comprises:
providing a perspective of trace information showing what application code triggered kernel event; and
differentiating traces caused by different codes.

12. The system of claim 9 , wherein said anomaly trace localization includes trace clustering comprising:
clustering traces by using program behavior features, said clustering using connectivity based clustering with a threshold in a distance function with a bottom-up approach and single-linkage being used to connect clusters.

13. The system of claim 9 , wherein said anomaly trace localization includes behavior feature matching comprising:
determining which clusters are comparable responsive to the first and second sets of input traces being clusters,
comparing traces from the first and second sets of input traces for clusters in a matched group;
representing traces generated by application code patterns that are different from a training run for unmatched clusters, the unmatched clusters being analyzed in an aggregated way of by way of manual specification; and
basing matching criteria of behavior features on a combination of system call distance and common kinds of system calls.

14. The system of claim 9 , wherein said anomaly trace localization includes conditional data mini

Assignees

• Nec Lab America Inc

Inventors

• Rhee Jungwhan
• Jiang Guofei
• Yoshihira Kenji
• Zhang Hui

Classifications

• G06F11/3636 Primary

  by tracing the execution of the program · CPC title

• G06F2201/86

  Event-based monitoring · CPC title

• G06F2201/865

  Monitoring of software · CPC title

• G06F11/34

  Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation {; Recording or statistical evaluation of user activity, e.g. usability assessment} · CPC title

• G06F11/3466

  Performance evaluation by tracing or monitoring · CPC title