Policy-based detection of anomalous control and data flow paths in an application program

Patent metadata

Field	Value
Publication number	US-10902121-B2
Application number	US-201715788473-A
Country	US
Kind code	B2
Filing date	Oct 19, 2017
Priority date	Oct 19, 2017
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

Anomalous control and data flow paths in a program are determined by machine learning the program's normal control flow paths and data flow paths. A subset of those paths also may be determined to involve sensitive data and/or computation. Learning involves collecting events as the program executes, and associating those event with metadata related to the flows. This information is used to train the system about normal paths versus anomalous paths, and sensitive paths versus non-sensitive paths. Training leads to development of a baseline “provenance” graph, which is evaluated to determine “sensitive” control or data flows in the “normal” operation. This process is enhanced by analyzing log data collected during runtime execution of the program against a policy to assign confidence values to the control and data flows. Using these confidence values, anomalous edges and/or paths with respect to the policy are identified to generate a “program execution” provenance graph associated with the policy.

First claim

Having described our invention, what we claim is as follows:

1. A method for detecting anomalous behavior of an application program, comprising:
receiving trace data generated from multiple invocations of the application program;
based at least in part on the received trace data, building a baseline provenance graph that models a normal control flow or data flow in the application program, and that identifies any path within a control flow or data flow that involves sensitive data or computation, wherein at least one edge in the baseline provenance graph has at least one or more first confidence values associated with a probability of that edge being traversed;
during runtime execution of the application program against a policy, wherein the policy is one of: a security policy, a compliance policy and a network policy, receiving log data;
using the received log data to assign second confidence values to at least one of the control or data flows with respect to the policy; and
identifying that the at least one edge is anomalous by comparing the assigned second confidence values with the at least one or more first confidence values, the edge identified as anomalous representing the anomalous behavior; and
responsive to detecting the anomalous behavior, taking a further corrective action.

2. The method as described in claim 1 further including building a program execution provenance graph associated with the policy and that includes the control or data flows and their assigned confidence values.

3. The method as described in claim 2 wherein the program execution provenance graph is built using machine learning.

4. The method as described in claim 3 further including adjusting a confidence value assigned to a given control or data flow based on the machine learning.

5. The method as described in claim 1 wherein the baseline provenance graph comprises a control flow graph, and a data flow graph.

6. The method as described in claim 2 further including reporting an application program behavior anomaly identified from the program execution provenance graph.

7. An apparatus for detecting anomalous behavior of an application program, comprising:
a processor;
computer memory holding computer program instructions executed by the processor, the computer program configured to:
receive trace data generated from multiple invocations of the application program;
based at least in part on the received trace data, build a baseline provenance graph that models a normal control flow or data flow in the application program, and that identifies any path within a control flow or data flow that involves sensitive data or computation, wherein at least one edge in the baseline provenance graph has at least one or more first confidence values associated with a probability of that edge being traversed;
during runtime execution of the application program against a policy, wherein the policy is one of: a security policy, a compliance policy and a network policy, receive log data;
use the received log data to assign second confidence values to at least one of the control or data flows with respect to the policy; and
identify that the at least one edge is anomalous by comparing the assigned second confidence values with the at least one or more first confidence values, the edge identified as anomalous representing the anomalous behavior; and
responsive to detecting the anomalous behavior, take a further corrective action.

8. The apparatus as described in claim 7 wherein the computer program instructions are further configured to build a program execution provenance graph associated with the policy and that includes the control or data flows and their assigned confidence values.

9. The apparatus as described in claim 8 wherein the program execution provenance graph is built using machine learning.

10. The apparatus as described in claim 9 wherein the computer program instructions are further configured to adjust a confidence value assigned to a given control or data flow based on the machine learning.

11. The apparatus as described in claim 7 wherein the baseline provenance graph comprises a control flow graph, and a data flow graph.

12. The apparatus as described in claim 8 wherein the computer program instructions are further configured to report an application program behavior anomaly identified from the program execution provenance graph.

13. A computer program product in a non-transitory computer readable medium for use in a data processing system for detecting anomalous behavior of an application program the computer program product holding computer program instructions that, when executed by the data processing system, are configured to:
receive trace data generated from multiple invocations of the application program;
based at least in part on the received trace data, build a baseline provenance graph that models a normal control flow or data flow in the application program, and that identifies any path within a control flow or data flow that involves sensitive data or computation, wherein at least one edge in the baseline provenance graph has at least one or more first confidence values associated with a probability of that edge being traversed;
during runtime execution of the application program against a policy, wherein the policy is one of: a security policy, a compliance policy and a network policy, receive log data;
use the received log data to assign second confidence values to at least one of the control or data flows with respect to the policy;
identify that the at least one edge is anomalous by comparing the assigned second confidence values with the at least one or more first confidence values, the edge identified as anomalous representing the anomalous behavior; and
responsive to detecting the anomalous behavior, take a further corrective action.

14. The computer program product as described in claim 13 wherein the computer program instructions are further configured to build a program execution provenance graph associated with the policy and that includes the control or data flows and their assigned confidence values.

15. The computer program product as described in claim 14 wherein the program execution provenance graph is built using machine learning.

16. The computer program product as described in claim 15 wherein the computer program instructions are further configured to adjust a confidence value assigned to a given control or data flow based on the machine learning.

17. The computer program product as described in claim 13 wherein the baseline provenance graph comprises a control flow graph, and a data flow graph.

18. The computer program product as described in claim 14 wherein the computer program instructions are further configured to report an application program behavior anomaly identified from the program execution provenance graph.

Assignees

• IBM

Inventors

• Chari Suresh
• Kundu Ashish
• Molloy Ian Michael
• Pendarakis Dimitrios

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06F2221/033

  Test or assess software · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F21/566 Primary

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title