Automated multi-phase investigation of security incident alerts using a large language model (LLM) with converging dialogue

Patent metadata

Field	Value
Publication number	US-12530469-B2
Application number	US-202418622946-A
Country	US
Kind code	B2
Filing date	Mar 31, 2024
Priority date	Mar 31, 2024
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

Automated multi-phase investigation of security incident alerts using a Large Language Model (LLM) with converging dialogue. A computerized system receives a Security Alert Message pertaining to a possible security-related incident pertaining to an organization. The system automatically evaluates whether the Security Alert Message is either (I) a False Positive security alert message or (II) a True Positive security alert message, by performing an iterative multi-phase converging process in which the LLM evaluates at least: (i) the content of that Security Alert Message, and (ii) the meta-data of that Security Alert Message, and (iii) organizational context that is related to that Security Alert Message. An iterative process is performed by the LLM, which utilizes an Agent Module to fetch additional context information from organizational sources. The LLM re-updates the Risk Score and re-evaluates the Risk Score until convergence to a decision.

First claim

What is claimed is:

1 . A computerized method comprising:
(a) receiving a Security Alert Message pertaining to a possible security-related incident pertaining to an organization; (b) automatically evaluating whether the Security Alert Message is either (I) a False Positive security alert message or (II) a True Positive security alert message,
by performing an iterative multi-phase converging process in which a Large Language Model (LLM) evaluates at least: (i) content of said Security Alert Message, and (ii) meta-data of said Security Alert Message, and (iii) organizational context that is related to said Security Alert Message;
(c) wherein as part of said iterative multi-phase converging process, said LLM iteratively determines and updates a Risk Score for said Security Alert Message by taking into account: (i) an alert-specific playbook that is generated automatically for said Security Alert Message, and (ii) the content of said Security Alert Message, and (iii) the meta-data of said Security Alert Message, and (iv) the organizational context that is related to said Security Alert Message; (d1) if a most-updated value of the Risk Score of said Security Alert Message is smaller than a pre-defined False Positive threshold value, then: marking said Security Alert Message as a False Positive alert message; (d2) if the most-updated value of the Risk Score of said Security Alert Message is greater than a pre-defined True Positive threshold value, then: marking said Security Alert Message as a True Positive alert message, and activating one or more pre-defined threat mitigation operations that are determined to be relevant to said Security Alert Message; (d3) if the most-updated value of the Risk Score of said Security Alert Message is not greater than said pre-defined True Positive threshold value, and is also not smaller than said pre-defined False Positive threshold value, then: automatically obtaining additional context information that is relevant to said Security Alert Message; automatically providing the additional context information to said LLM; prompting the LLM to update the Risk Score based on said additional context information; iteratively re-evaluating the Risk Score as (i) indicating a False Positive alert message, or as (ii) indicating a True Positive alert message, or (iii) being inconclusive and requiring at least one additional iteration of obtaining additional context data; (e) wherein obtaining the additional context information comprises:
generating a follow-up question that is addressed to a particular user in the organization;
sending said follow-up question to said particular user;
receiving an answer from said particular user to said follow-up question;
utilizing said answer as additional context by said LLM for updating the Risk Score for said Security Alert Message;
(f) prior to generating the follow-up question to said particular user, performing at least one of: (f1) estimating by the LLM whether or not any additional context data, that would be relevant for evaluating said Security Alert Message, can be obtained from non-human information sources that are available to the organization; if, and only if, the LLM estimates that no additional context data, that would be relevant for evaluating said Security Alert Message, can be obtained from non-human information sources that are available to the organization, then generating the follow-up question to said particular user; (f2) checking by the LLM whether or not any follow-up questions, that were already posed to one or more users in said organization with regard to other security alert messages, and their respective answers as stored in a questions-and-answers database, are useful for updating the Risk Score for said Security Alert Message; if, and only if, the checking result is negative, then: generating the follow-up question to said particular user;
wherein operations of said computerized method are implemented by utilizing at least: a hardware processor that is configured to electronically execute code, and a memory unit that is configured to electronically store code and to electronically store data.

2 . The computerized method of claim 1 ,
wherein obtaining the additional context information comprises: prompting the LLM to generate a follow-up question that the LLM estimates to trigger a response that can provide new useful context for re-evaluating said Security Alert Message; based on said follow-up questions, instructing an automated Agent Unit to fetch a particular information item from an Organizational Context Database as a response to said follow-up question.

3 . The computerized method of claim 1 ,
further comprising: storing in a questions-and-answers database the follow-up question and the information item that the automated Agent Unit fetched from the Organizational Context Database, and re-using said particular information item for evaluating a subsequent, different, security alert message; and performing Retrieval-Augmented Generation (RAG) of prompts, that are fed into said LLM, based on content that was accumulated in said questions-and-answers database in previous automated investigations of previous alert messages.

4 . The computerized method of claim 1 ,
further comprising: storing in a questions-and-answers database the follow-up question item and said answer, and re-using said follow-up question and said answer for evaluating a subsequent, different, security alert message.

5 . The computerized method of claim 1 , comprising:
prompting said LLM to update the Risk Score for said Security Alert Message, based on organizational context information that includes at least: (i) user data and account data from an Active Directory; (ii) data from organizational file systems and organizational data repositories; (iii) data from event logs of said organization.

6 . The computerized method of claim 1 ,
wherein the organizational context information further comprises: organizational context that was automatically calculated and derived from organizational data, and that describes Peer Groups among users of said organizations, generated using a peer group detection process that checks which users regularly interact with which other users in said organization.

7 . The computerized method of claim 1 ,
wherein the organizational context information further comprises: organizational context that describes at least: (i) users in said organization, and access privileges of each user to each resource of the organization; (ii) devices in said organization, and access privileges of each device to each resource of the organization.

8 . The computerized method of claim 1 ,
wherein the organizational context information further comprises: additional organizational context that is extracted from one or more of:
a Customer Relationship Management (CRM) system of said organization,
a Supply Chain Management (SCM) system of said organization,
an Enterprise Resource Planning (ERP) system of said organization,
an Active Directory of said organization.

9 . The computerized method of claim 1 , comprising:
fine-tuning said LLM, by modifying weights of parameters that said LLM uses, based on a specific dataset of correctly-evaluated security alert messages and their content and their meta-data.

10 . The computerized method of claim 1 ,
comprising: fine-tuning the LLM to generate relevant and accurate outputs in response to engineered prompts that command said LLM to perform a specific task of generating a Risk Score for an incoming Security Alert Message in a specific field of Security Domain.

11 . The computerized method of claim 1 ,
comprising: it

Assignees

• Varonis Systems Inc

Inventors

• Belgi Amir
• Sneh Ron
• Chen Lior

Classifications

• G06F2221/034

  Test or assess a computer or a system · CPC title

• G06F21/577 Primary

  Assessing vulnerabilities and evaluating computer system security · CPC title