System and method for threat detection based on stack trace and kernel sensors

Patent metadata

Field	Value
Publication number	US-12530449-B2
Application number	US-202218148965-A
Country	US
Kind code	B2
Filing date	Dec 30, 2022
Priority date	Dec 30, 2022
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

A method and system for threat detection and analysis is disclosed herein. The method includes monitoring at least one thread associated with at least one process on a computing system. The method further includes detecting specific system calls associated with at least one process at kernel level. The specific system calls are analyzed by applying a filter to the system calls sequence feature set associated with the specific system calls for detecting one or more events of interest. A full stack trace capture of at least one process is requested if the system calls sequence feature set is filtered and the one or more events of interest are detected. A first level monitoring is provided to the computing system, which includes processing and analyzing the captured full stack trace by a machine learning (ML) stack trace analyzer to generate a first verdict for threat detection and analysis.

First claim

The invention claimed is:

1 . A method for threat detection and analysis in a computer system with a kernel level, the method comprising:
training a machine learning (ML) stack trace analyzer on pairs of historical stack traces and system calls to output a threat prediction for an input convolution of a stack trace; monitoring threads of a first process on the computing system; detecting specific system calls corresponding to the first process at the kernel level; analyzing the specific system calls by applying a filter to a system-calls sequence feature set associated with the specific system calls for detecting a sequence of events of interest, the events of interest comprising a plurality of specific system calls; requesting a full stack trace capture of the first process when the sequence of events of interest is detected; providing a first level monitoring to the computing system, wherein the first level monitoring comprises processing the captured full stack trace to create a convolution of the full stack trace and analyzing the convolution of the captured full stack trace by the ML stack trace analyzer to output a threat prediction; generating a probabilistic first verdict based on the threat prediction for threat detection and analysis; and taking a response action in accordance with the first verdict.

2 . The method of claim 1 , wherein the response action comprises at least one of terminating the process, freezing the process, killing the process, quarantining the process, determining the file associated with the source process and deleting the file, or determining the network connections associated with the target or source processes and blocking the connections.

3 . The method of claim 1 , further comprising capturing a call stack trace at kernel level associated with the at least one process.

4 . The method of claim 1 , further comprising providing a second level monitoring to the computing system, wherein the second level monitoring includes providing the first verdict and the captured call stack to an aggregated ML analyzer to generate a second verdict for threat detection and analysis.

5 . The method of claim 1 , further comprising the steps of:
monitoring threads of a second process on a computing system; detecting a sequence of specific system calls corresponding to the second process at kernel level; and associating the detected system calls of the first and the second processes.

6 . The method of claim 5 , wherein the first process is a target process, and the second process is a source process.

7 . The method of claim 6 , further comprising determining the source process based on associated system calls in response to the generated verdict.

8 . The method of claim 7 , further comprising analyzing the source process with static and dynamic analyzers for threat detection.

9 . The method of claim 1 , wherein the requesting the full stack trace capture is performed by a file protection driver.

10 . The method of claim 3 , further comprising preprocessing the full stack trace by:
filtering whitelist calls from the full stack trace; and deduplicating calls.

11 . The method of claim 1 , wherein a stack trace is not collected independently from a detected sequence of events of interest.

12 . The method of claim 11 , wherein the detected sequence of events of interest comprises a sequence of at least four system calls.

13 . A system for threat detection and analysis, the system comprising:
a processor coupled to memory and nonvolatile storage, the processor running an operating system comprising user space and kernel space, the kernel space comprising a kernel sensor configured to:
monitor threads of a first process on a computing system;
detect specific system calls corresponding to the first process at kernel level;
analyze the specific system calls by applying a filter to system-calls sequence feature set associated with the specific system calls for detecting a sequence of events of interest, the events of interest comprising a plurality of specific system calls; and
request a full stack trace capture of the first process when the sequence of events of interest is detected;
a file protection driver, configured to capture requested full stack trace; and
a machine learning (ML) stack trace analyzer configured to provide a first level monitoring to the computing system, wherein the ML stack trace analyzer has been trained on pairs of historical stack traces and system calls to output a threat prediction for an input convolution of a stack trace, and wherein the first level monitoring includes processing the captured full stack trace to create a convolution of the full stack trace and analyzing the convolution of the captured full stack trace to output a threat prediction and generating a probabilistic first verdict based on the threat prediction for threat detection and analysis.

14 . The system of claim 13 , wherein the kernel sensor is further configured to capture a call stack trace associated with at least one process at the kernel level.

15 . The system of claim 13 , further comprising an aggregated ML analyzer configured to provide a second level monitoring to the computing system, wherein the second level monitoring includes providing the first verdict and the captured call stack trace to the call stack ML analyzer and generating a second verdict for threat detection and analysis.

16 . The system of claim 13 , wherein the kernel sensor is further configured to:
monitor threads of the second process on a computing system; detect a sequence of specific system calls corresponding to the second process at kernel level; and associate detected system calls of the first and the second processes.

17 . The system of claim 16 , wherein the first process is a target process, and the second process is a source process.

18 . The system of claim 17 , further comprising determining the source process based on associated system calls in response to the generated verdict.

19 . The system of claim 18 , further comprising static and dynamic analyzers, configured to analyze the source process.

20 . The system of claim 13 , wherein the file protection driver is not configured to collect stack traces independently from the detected sequence of events of interest.

Assignees

• Acronis Int Gmbh

Inventors

• Strogov Vladimir
• Ulasen Sergey
• Dodz Aliaksei
• Bell Serg
• Protasov Stanislav

Classifications

• G06F2221/034

  Test or assess a computer or a system · CPC title

• G06F21/566

  Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities · CPC title

• G06F21/554 Primary

  involving event detection and direct action · CPC title