Path scanning for the detection of anomalous subgraphs and use of DNS requests and host agents for anomaly/change detection and network situational awareness

Patent metadata

Field	Value
Publication number	US-10243984-B2
Application number	US-201715809297-A
Country	US
Kind code	B2
Filing date	Nov 10, 2017
Priority date	Mar 22, 2012
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

A system, apparatus, computer-readable medium, and computer-implemented method are provided for detecting anomalous behavior in a network. Historical parameters of the network are determined in order to determine normal activity levels. A plurality of paths in the network are enumerated as part of a graph representing the network, where each computing system in the network may be a node in the graph and the sequence of connections between two computing systems may be a directed edge in the graph. A statistical model is applied to the plurality of paths in the graph on a sliding window basis to detect anomalous behavior. Data collected by a Unified Host Collection Agent (“UHCA”) may also be used to detect anomalous behavior.

First claim

The invention claimed is:

1. A computer-implemented method, comprising:
applying, by a computing system, an edge resolution model to a plurality of enumerated k-paths on a sliding window basis; and
detecting, by the computing system, anomalous behavior based on the applied edge resolution model, wherein
the edge resolution model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

2. The computer-implemented method of claim 1 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

3. The computer-implemented method of claim 1 , further comprising:
determining, by the computing system, historical parameters of the network to determine normal activity levels, wherein
the computing system determines the historical parameters by taking into account at least two edge types.

4. The computer-implemented method of claim 3 , wherein a first edge type comprises member edges having sufficient data to estimate an individual model, and a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

5. The computer-implemented method of claim 4 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

6. The computer-implemented method of claim 1 , further comprising:
collecting data, by the computing system, from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network; and
analyzing the collected data, by the computing system, to detect anomalous behavior during a predetermined time period.

7. An apparatus, comprising:
at least one processor; and
memory storing computer program instructions, wherein the instructions, when executed by the at least one processor, are configured to cause the at least one processor to:
apply a statistical model to a plurality of enumerated k-paths on a sliding window basis, and
detect anomalous behavior based on the applied statistical model, wherein

the statistical model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

8. The apparatus of claim 7 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

9. The apparatus of claim 7 , wherein the computer program instructions are further configured to cause the at least one processor to determine historical parameters of the network to determine normal activity levels by taking into account at least two edge types.

10. The apparatus of claim 9 , wherein a first edge type comprises member edges having sufficient data to estimate an individual model, and a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

11. The apparatus of claim 10 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

12. The apparatus of claim 7 , wherein the computer program instructions are further configured to cause the at least one processor to:
collect data from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network, and
analyze the collected data to detect anomalous behavior during a predetermined time period.

13. A system, comprising:
memory storing computer program instructions; and
a plurality of processing cores configured to execute the stored computer program instructions, wherein the plurality of processing cores is configured to:
apply a statistical model to a plurality of enumerated k-paths on a sliding window basis, and
detect anomalous behavior based on the applied statistical model, wherein

the statistical model comprises an Observed Markov Model (“OMM”) or a Hidden Markov Model (“HMM”).

14. The system of claim 13 , wherein
the OMM or the HMM comprise two-state models,
an “on” state indicates user presence, and
an “off” state indicates that the user is not present.

15. The system of claim 13 , wherein
the plurality of processing cores is further configured to determine historical parameters by taking into account at least two edge types,
a first edge type comprises member edges having sufficient data to estimate an individual model, and
a second edge type comprises member edges where there is not sufficient data to estimate individual models for the member edges.

16. The system of claim 15 , wherein the second edge type is parameterized by a mean vector to ensure that models are not overly sensitive to low count edges.

17. The system of claim 13 , wherein the plurality of processing cores is further configured to:
collect data from a plurality of host agents pertaining to network communications sent and received by respective hosts in the network, and
analyze the collected data to detect anomalous behavior during a predetermined time period.

18. A computer-implemented method, comprising:
analyzing, by the computing system, collected data pertaining to network communications for each host of a plurality of hosts in a network to detect anomalous behavior during a predetermined time period by applying a statistical model to a plurality of enumerated k-paths on a sliding window basis; and
when anomalous behavior is detected, providing, by the computing system, an indication that the anomalous behavior occurred during the predetermined time period, wherein
the collected data is sent as one-way communications from the host agents via User Datagram Protocol (“UDP”).

19. The computer-implemented method of claim 18 , wherein the data collected for each host comprises process stop and start information with checksums of starting process images, network connection event logs, a mapping of running processes to established network connections, and a current network connection state.

20. The computer-implemented method of claim 18 , wherein the collected data comprises a list of triples of values indicating network communication between hosts, each triple comprising a time when the communication occurred, a source Internet Protocol (“IP”) address, and a destination IP address.

21. The computer-implemented method of claim 18 , wherein the collecting of the data further comprises periodically polling the host agents for the data.

22. The computer-implemented method of claim 18 , further comprising:
using, by the computing system, a Transmission Control Protocol (“TCP”) time wait state to collect information on short duration connections.

23. The computer-implemented method of claim 18 , further comprising:
establishing, by the computing system, count weights using count information by calculating mean and variance statistics on counts.

24. The computer-implemented method of claim 18 , wherein the data is collected proportionally to a level of anomalousness on a respective host,
at a low level of anomalousness, as deemed by deviation from a baseline probabilistic approach, the computing system collects basic network connectivity and process information,
at a moderate level of anomalousness, the computing system collects more process accounting and services and more complete network behavioral data, and
at a high level of anomalousness, the computing system collects full host behavio

Assignees

• Triad Nat Security Llc

Inventors

• Neil Joshua Charles
• Fisk Michael Edward
• Brugh Alexander William
• Hash Jr Curtis Lee
• Storlie Curtis Byron
• Uphoff Benjamin
• Kent Alexander

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N5/022

  Knowledge engineering; Knowledge acquisition · CPC title

• G06N5/045

  Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence · CPC title

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title

• H04L2463/144

  Detection or countermeasures against botnets · CPC title