Path scanning for the detection of anomalous subgraphs and use of DNS requests and host agents for anomaly/change detection and network situational awareness

Patent metadata

Field	Value
Publication number	US-9825979-B2
Application number	US-201715419673-A
Country	US
Kind code	B2
Filing date	Jan 30, 2017
Priority date	Mar 22, 2012
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

A system, apparatus, computer-readable medium, and computer-implemented method are provided for detecting anomalous behavior in a network. Historical parameters of the network are determined in order to determine normal activity levels. A plurality of paths in the network are enumerated as part of a graph representing the network, where each computing system in the network may be a node in the graph and the sequence of connections between two computing systems may be a directed edge in the graph. A statistical model is applied to the plurality of paths in the graph on a sliding window basis to detect anomalous behavior. Data collected by a Unified Host Collection Agent (“UHCA”) may also be used to detect anomalous behavior.

First claim

The invention claimed is:

1. A computer-implemented method, comprising:
enumerating, by a computing system, a plurality of k-paths in the network as part of a graph representing a network;
applying, by the computing system, a Markov edge resolution model to the plurality of k-paths in the graph on a sliding window basis; and
detecting, by the computing system, anomalous behavior based on the applied Markov edge resolution model.

2. The computer-implemented method of claim 1 , wherein the Markov edge resolution model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

3. The computer-implemented method of claim 2 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

4. The computer-implemented method of claim 1 , further comprising:
determining, by the computing system, historical parameters of the network to determine normal activity levels, wherein
the computing system determines the historical parameters by taking into account at least two edge types.

5. The computer-implemented method of claim 4 , wherein a first edge type comprises member edges having sufficient data to estimate an individual model, and a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

6. The computer-implemented method of claim 5 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

7. The computer-implemented method of claim 1 , further comprising:
collecting data, by the computing system, from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network; and
analyzing the collected data to detect anomalous behavior during a predetermined time period.

8. An apparatus, comprising:
at least one processor; and
memory storing computer program instructions, wherein the instructions, when executed by the at least one processor, are configured to cause the at least one processor to:
enumerate a plurality of k-paths in the network as part of a graph representing the network,
apply a statistical model to the plurality of k-paths in the graph on a sliding window basis, and
detect anomalous behavior based on the applied statistical model.

9. The apparatus of claim 8 , wherein the statistical model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

10. The apparatus of claim 8 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

11. The apparatus of claim 8 , wherein the computer program instructions are further configured to cause the at least one processor to determine historical parameters of a network to determine normal activity levels by taking into account at least two edge types.

12. The apparatus of claim 11 , wherein a first edge type comprises member edges having sufficient data to estimate an individual model, and a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

13. The apparatus of claim 12 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

14. The apparatus of claim 8 , wherein the computer program instructions are further configured to cause the at least one processor to:
collect data from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network, and
analyze the collected data to detect anomalous behavior during a predetermined time period.

15. A system, comprising:
memory storing computer program instructions configured to detect anomalous behavior in a network; and
a plurality of processing cores configured to execute the stored computer program instructions, wherein the plurality of processing cores is configured to:
enumerate a plurality of k-paths in the network as part of a graph representing the network,
apply a statistical model to the plurality of k-paths in the graph on a sliding window basis, and
detect anomalous behavior based on the applied statistical model.

16. The system of claim 15 , wherein the statistical model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

17. The system of claim 16 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

18. The system of claim 15 , wherein
the plurality of processing cores is further configured to determine historical parameters by taking into account at least two edge types,
a first edge type comprises member edges having sufficient data to estimate an individual model, and
a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

19. The system of claim 18 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

20. The system of claim 15 , wherein the plurality of processing cores is further configured to:
collect data from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network, and
analyze the collected data to detect anomalous behavior during a predetermined time period.

21. A computer-implemented method, comprising:
analyzing, by the computing system, collected data for each host of a plurality of hosts pertaining to network communications to detect anomalous behavior during a predetermined time period by applying a statistical model to a plurality of k-paths in a graph on a sliding window basis; and
when anomalous behavior is detected, providing, by the computing system, an indication that the anomalous behavior occurred during the predetermined time period.

22. The computer-implemented method of claim 21 , wherein the collected data is sent as one-way communications from the host agents via User Datagram Protocol (“UDP”).

23. The computer-implemented method of claim 21 , wherein the data collected for each host comprises process stop and start information with checksums of starting process images, network connection event logs, a mapping of running processes to established network connections, and a current network connection state.

24. The computer-implemented method of claim 21 , wherein the collected data comprises a list of triples of values indicating network communication between hosts, each triple comprising a time when the communication occurred, a source Internet Protocol (“IP”) address, and a destination IP address.

25. The computer-implemented method of claim 21 , wherein the collecting of the data further comprises periodically polling the host agents for the data.

26. The computer-implemented method of claim 21 , further comprising:
using, by the computing system, a Transmission Control Protocol (“TCP”) time wait state to collect information on short duration connections.

27. The computer-implemented method of claim 21 , further comprising:
establishing, by the computing system, count weights using count information by calculating mean and variance statistics on c

Assignees

• Los Alamos Nat Security Llc

Inventors

• Neil Joshua Charles
• Fisk Michael Edward
• Brugh Alexander William
• Hash Curtis Lee
• Storlie Curtis Byron
• Uphoff Benjamin
• Kent Alexander

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• H04L63/1408

  by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title