Systems and methods for anomaly detection

Patent metadata

Field	Value
Publication number	US-10917419-B2
Application number	US-201715587871-A
Country	US
Kind code	B2
Filing date	May 5, 2017
Priority date	May 5, 2017
Publication date	Feb 9, 2021
Grant date	Feb 9, 2021

Abstract

An anomaly detection module may include a time-series analyzer that classifies current time-series data into at least one of a plurality of classifications based upon historical data and may construct a statistical model representative of the current time-series data based upon the at least one of the plurality of classifications. An anomaly detector monitors a stream of the current time-series data and identifies statistical outliers of the stream of the current time-series data, based upon the statistical model and may determine an anomalous score for the statistical outliers by tracking a history of the statistical outliers; wherein the anomalous score comprises a representation of a magnitude of deviation between the current time-series data and the statistical model over multiple measurements of the current time-series data, over a particular time interval, or both.

First claim

What is claimed is:

1. A system for analyzing an information technology network, comprising:
a time-series analyzer implemented as instructions executed by one or more processors, wherein the time-series analyzer is configured to:
classify a stream of current time-series data for the information technology network into at least one of a plurality of classifications based at least in part upon historical time-series data; and
construct a statistical model representative of the stream of current time-series data based upon the at least one of the plurality of classifications; and

an anomaly detector implemented as instructions executed by the one or more processors, wherein the anomaly detector is configured to:
monitor the stream of current time-series data;
perform a statistical analysis on the stream of current time-series data based at least in part upon the statistical model constructed by the time-series analyzer;
identify statistical outliers of the stream of current time-series data based at least on the statistical analysis;
determine an anomalous score for the statistical outliers based at least on a history of the statistical outliers, wherein the anomalous score is based at least in part on a magnitude and duration of deviation between the current time-series data and the statistical model; and
in response to determining that the anomalous score is above a pre-determined threshold;
generate an event corresponding to the anomalous score;
transmit the event to an alerts data store communicatively coupled to the system; and
schedule remedial actions comprising hardware replacement based on the event.

2. The system of claim 1 , wherein the plurality of classifications comprise:
a quasi-normal classification, indicating that the current time-series data appears to adhere to a stationary process;
a near-constant classification, indicating that the current time-series data appears to adhere to a constant value;
a categorical classification, indicating that the current time-series data appears to adhere to a process having a set of discrete values;
a switching classification, indicating that the current time-series data appears to adhere to a process of jumping from at least one first state to at least one second state via one or more state transitions; or
an arrival classification, indicating that the current time-series data appears to adhere to a process dependent upon a timing of a particular event.

3. The system of claim 2 , wherein the plurality of classifications comprises an other classification, indicating that the stream of current time-series data is not classified as the quasi-normal classification, the near-constant classification, the categorical classification, the switching classification, or the arrival classification.

4. The system of claim 3 , wherein the time-series analyzer is configured to construct the statistical model by:
when the stream of current time-series data is classified under the quasi-normal classification, characterizing the stream of current time-series data using one or more broad bands based upon a standard deviation of the historical time-series data.

5. The system of claim 2 , wherein the time-series analyzer is configured to construct the statistical model by:
when the stream of current time-series data is classified under the quasi-normal classification, characterizing the stream of current time-series data using a mean and standard deviation of the stream of current time-series data;
when the stream of current time-series data is classified under the near-constant classification, characterizing the stream of current time-series data using the constant value;
when the stream of current time-series data is classified under the categorical classification, characterizing the stream of current time-series data using a categorical probability distribution;
when the stream of current time-series data is classified under the switching classification, characterizing the stream of current time-series data using one or more statistics of the at least one first state, the at least one second state, or both and one or more statistics of the one or more state transitions; and
when the stream of current time-series data is classified under the arrival classification, characterizing the stream of current time-series data based at least in part upon the timing of the particular event.

6. The system of claim 1 , wherein the plurality of classifications comprise a seasonal classification, indicating that the current time-series data appears to adhere to a process having periodic non-normal distribution and a general trend over time, and wherein when the stream of current time-series data is classified under the seasonal classification, the time-series analyzer is configured to construct the statistical model by characterizing the stream of current time-series data by:
extracting the periodic non-normal distribution and the general trend over time from the stream of current time-series data;
smoothing the periodic non-normal distribution and the general trend over time;
fitting the smoothed periodic non-normal distribution with a spline that has on the order of one knot per hour; and
extrapolating the general trend as a constant or as a linear trend.

7. The system of claim 1 , wherein the anomaly detector is configured to:
query the statistical model to determine upper and lower control limits based at least in part upon a projection of an expected trend and seasonality for subsequent time-series data that is provided by the statistical model;
wherein the statistical outliers are identified as data outside of a range defined by the upper and lower control limits.

8. The system of claim 7 , wherein the anomaly detector is configured to determine the anomalous score for the statistical outliers by:
calculating an estimate of a probability that the stream of current time-series data is in a normal range;
calculating an accumulated anomalousness measure, comprising an estimate of the integral of excess Z-Score attenuated by a time-windowing function that provides a measure of area between the stream of current outlier time-series data and the upper and lower control limits, when the time-series data is outside the range defined by the upper and lower control limits, wherein a window of the time-windowing function is defined based at least in part upon the estimate of the probability that the stream of current time-series data is in a normal range; and
applying a sigmoid function to the accumulated anomalousness measure.

9. The system of claim 8 , wherein the anomaly detector is configured to determine the anomalous score for the statistical outliers by mapping results of the application of the sigmoid function to a pre-determined scale for anomalous scores, resulting in the anomalous score.

10. A tangible, non-transitory, machine-readable medium, comprising machine-readable instructions to cause one or more processors to analyze an information technology network using steps comprising:
at periodic intervals, classify a stream of current time-series data for the information technology network into at least one of a plurality of classifications based at least in part upon historical time-series data;
construct a statistical model representative of the stream of current time-series data based upon the at least one of the plurality of classifications;
monitor the stream of current time-series data;
perform a statistical analysis on the stream of current time-series data based at least in part upon the statistical model;
identify statistical outliers of the stream of current time-series data based at least on the statistical analysis;
determine an an

Assignees

• Servicenow Inc

Inventors

• Crotinger James
• Tucker Scott
• Polinati Chinna Babu

Classifications

• G06F18/2433

  Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection · CPC title

• H04L63/1433

  Vulnerability analysis · CPC title

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title