System and method for real-time detection of anomalies in database usage

Patent metadata

Field	Value
Publication number	US-10409665-B2
Application number	US-201514732162-A
Country	US
Kind code	B2
Filing date	Jun 5, 2015
Priority date	Jun 9, 2014
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A system and method for real-time detection of anomalies in database or application usage is disclosed. Embodiments provide a mechanism to detect anomalies in database or application usage, such as data exfiltration attempts, first by identifying correlations (e.g., patterns of normalcy) in events across different heterogeneous data streams (such as those associated with ordinary, authorized and benign database usage, workstation usage, user behavior or application usage) and second by identifying deviations/anomalies from these patterns of normalcy across data streams in real-time as data is being accessed. An alert is issued upon detection of an anomaly, wherein a type of alert is determined based on a characteristic of the detected anomaly.

First claim

The invention claimed is:

1. A computerized system for real-time detection of anomalies occurring in an enterprise computer network, comprising:
at least one processor; and
at least one memory including program code that, when executed by the at least one processor, causes the system to:
receive a plurality of heterogeneous data streams from sources in the network, the sources including two levels, first level sources and second level sources,
wherein the first level sources include one or more selected from a group consisting of agents located at databases; agents located at applications; audit programs located at user workstations; sensors located in the network; and sensors located at access points to the network;
wherein the second level sources include one or more selected from a group consisting of data access, user behavior, computer activity and network activity; and
wherein the first level sources monitor event streams of the second level sources and generate data streams indicative of corresponding second level source activity in a uniform format;
process the heterogeneous data streams obtained by combining at least two of the first level sources to identify events therein, each event being identified by at least a unique ID, a timestamp, and an event type, wherein the processing of the heterogeneous data streams includes combining at least two of the first level sources into a single data stream;
correlate the processed heterogeneous data streams to form an integrated data stream comprising a plurality of identified events;
detect the existence and at least one characteristic of an anomaly in the computer network by application of a predetermined model of normalcy and one or more anomaly rules to the integrated data stream comprising the plurality of identified events; and
issue an alert based on the at least one characteristic of the anomaly.

2. The computerized system of claim 1 , wherein the executed program code causes the system to create the predetermined model of normalcy and the one or more anomaly rules by:
receiving additional data comprising the plurality of heterogeneous data streams, wherein the additional data corresponds to authorized and benign usage of network resources;
processing the heterogeneous data streams to identify events therein, each even being identified by at least a unique ID, a timestamp, and an event type;
correlating the processed data streams to form an integrated data stream comprising a plurality of identified events;
identifying one or more patterns from relations between identified events comprising the integrated data stream; and
creating the model of normalcy and the one or more anomaly rules based on the identified one or more patterns.

3. The computerized system of claim 2 , wherein the additional data corresponds to an ordinary, authorized, and benign database query and an ordinary, authorized, and benign user interaction at a user workstation.

4. The computerized system of claim 3 , wherein the predetermined model of normalcy comprises the user interaction at the user workstation within a predetermined period of time before the database query.

5. The computerized system of claim 3 , wherein the detected anomaly comprises a lack of user interaction at the user workstation within a predetermined period of time before the database query.

6. The computerized system of claim 1 , wherein the one or more anomaly rules relate to at least one of: how and whether anomalies are detected, how a detected anomaly is treated and characterized, and what reaction to employ in response to the detected anomaly.

7. The computerized system of claim 1 , wherein the detected anomaly is indicative of unauthorized manipulation or falsification of data, sabotage of a database, or exfiltration of data.

8. The computerized system of claim 1 , wherein the heterogeneous data streams comprise multi-modal asynchronous signals.

9. The computerized system of claim 1 , wherein the alert comprises at least one of an alarm message, a communication triggering further analysis and/or action, a command instructing the restriction or shutting down of an affected workstation, database, network or network access, initiation of additional targeted monitoring, analysis, and/or applications to capture additional detailed information regarding an attack, continued monitoring of a user, placement of a flag in a file for further follow-up, restricting access to a network, alerting security, and restricting or locking down a building or a portion of a building.

10. The computerized system of claim 1 , wherein the program code includes an algorithm that detects and extracts persistent events among the plurality of identified events in at least one of the plurality of heterogeneous data streams, and wherein the persistent events are time-stamped data that appear regularly overtime.

11. The computerized system of claim 10 , wherein the persistent events appear in different distributed streams among the plurality of heterogeneous data streams.

12. The computerized system of claim 10 , wherein the at least one of the plurality of heterogeneous data streams is statistically sampled to reduce stream size of the at least one of the plurality of heterogeneous data streams, without overlooking the persistent events.

13. The computerized system of claim 1 , wherein the processing of the heterogeneous data streams includes transformation of heterogeneous data stream inputs into a standardized data structure to produce homomorphism.

14. The computerized system of claim 1 , wherein the processing of the heterogeneous data streams includes using an automatic event tabulator and correlator.

15. The computerized system of claim 1 , wherein the processing of the heterogeneous data streams includes operating on the single data stream using an algorithm that identifies temporal relationships.

16. The computerized system of claim 1 , wherein the processing of the heterogeneous data streams includes operating on the single data stream using an algorithm that identifies spatial relationships.

17. A computerized system for real-time detection of anomalies occurring in an enterprise computer network, comprising:
at least one processor; and
at least one memory including program code that, when executed by the at least one processor, causes the system to:
receive a plurality of heterogeneous data streams from sources in the network, the sources including two levels, first level sources and second level sources,
wherein the first level sources include the following: agents located at databases; agents located at applications; audit programs located at user workstations; sensors located in the network; and sensors located at access points to the network; and
wherein the second level sources include the following: email exchanges; instant messages; voice mails; documents; database content; and user interactions and database queries;
process the heterogeneous data streams obtained by combining at least one of the first level sources with at least one of the second level sources to identify events therein, each event being identified by at least a unique ID, a timestamp, and an event type;
correlate the processed heterogeneous data streams to form an integrated data stream comprising a plurality of identified events;
detect the existence and at least one characteristic of an anomaly in the computer network by application of a predetermined model of normalcy and one or more anomaly rules to the integrated data stream comprising the plurality of identified events; and issue an alert base

Assignees

• Northrop Grumman Systems Corp
• Northrup Grumman Systems Corp

Inventors

• Steiner Donald
• Day John

Classifications

• G06F21/6227

  where protection concerns the structure of data, e.g. records, types, queries · CPC title

• G06F11/0727

  in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F21/552

  involving long-term monitoring or reporting · CPC title

• G06F11/3438

  monitoring of user actions (tracking the activity of the user H04L67/535) · CPC title