Systems and methods for automated rule-based detection

Patent metadata

Field	Value
Publication number	US-12182255-B2
Application number	US-202217589773-A
Country	US
Kind code	B2
Filing date	Jan 31, 2022
Priority date	Jan 31, 2022
Publication date	Dec 31, 2024
Grant date	Dec 31, 2024

Abstract

Systems and methods for generating, selecting, and implementing rule-based strategies are disclosed. An input data set representing a plurality of interactions that may be classified as malicious or non-malicious is received and at least one strategy tree including a plurality of rule-based strategies is generated. The at least one strategy trees is generated by a machine learning model configured to generate a tree structure. The rule-based strategies are ranked based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies and at least a first rule-based strategy having a highest PRS score is extracted. One or more interactions are evaluated using the first rule-based strategy to determine when the one or more interactions are malicious.

First claim

What is claimed is:

1. A system, comprising:
a non-transitory memory having instructions stored thereon and a processor configured to read the instructions to:
receive an input data set representing a plurality of interactions that may be classified as malicious or non-malicious;
generate at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse false positive curve and trees is generated by a machine learning model configured to generate a tree structure;
rank the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy;
extract at least a first rule-based strategy having a highest PRS score; and
evaluate one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious.

2. The system of claim 1 , wherein the PRS score is determined according to:

PRS
=

3

1

α
*
Precision

+

1

β
*
Recall

+

1

γ
*
Stability

where each of α, β, and γ are independent weighting constants.

3. The system of claim 2 , wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy.

4. The system of claim 2 , wherein Recall is a ratio of true positive identifications to a total number of malicious interactions in an input data set.

5. The system of claim 2 , wherein Stability is determined based on variations in Precision and Recall over time.

6. The system of claim 1 , wherein the first rule-based strategy is implemented in a real-time decision engine language.

7. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor cause a device to perform operations comprising:
receiving an input data set representing a plurality of interactions that may be classified as malicious or non-malicious;
generating at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse false positive curve and is generated by a machine learning model configured to generate a tree structure;
ranking the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy;
extracting at least a first rule-based strategy having a highest PRS score; and
evaluating one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious.

8. The non-transitory computer readable medium of claim 7 , wherein the PRS score is determined according to:

PRS
=

3

1

α
*
Precision

+

1

β
*
Recall

+

1

γ
*
Stability

where each of α, β, and γ are independent weighting constants.

9. The non-transitory computer readable medium of claim 8 , wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy.

10. The non-transitory computer readable medium of claim 8 , wherein Recall is a ratio of true positive identifications to a total number of malicious interactions in an input data set.

11. The non-transitory computer readable medium of claim 8 , wherein Stability is determined based on variations in Precision and Recall over time.

12. The non-transitory computer readable medium of claim 7 , wherein the first rule-based strategy is implemented in a real-time decision engine language.

13. A method, comprising:
receiving an input data set representing a plurality of interactions that may be classified as malicious or non-malicious;
generating at least one strategy tree including a plurality of rule-based strategies, wherein the at least one strategy tree comprises a strategy tree having a highest area-under-the-curve metric among a plurality of strategy trees for a true positive verse fake positive curve and is generated by a machine learning model configured to generate a tree structure;
ranking the rule-based strategies based on a precision-recall-stability (PRS) score generated for each of the rule-based strategies, wherein the PRS score is a single metric combining Precision, Recall and Stability of each rule-based strategy;
extracting at least a first rule-based strategy having a highest PRS score; and
evaluating one or more interactions using the first rule-based strategy to determine when the one or more interactions are malicious.

14. The method of claim 13 , wherein the PRS score is determined according to:

PRS
=

3

1

α
*
Precision

+

1

β
*
Recall

+

1

γ
*
Stability

where each of α, β, and γ are independent weighting constants.

15. The method of claim 14 , wherein Precision is a ratio of true positive identifications to a total number of positive identifications of a selected rule-based strategy, Recall is a ratio of true positive identif

Assignees

• Walmart Apollo Llc

Inventors

• Zhou Siyuan
• Xia Jing
• Chobanyan Alexander
• Leonard Bradley Lamarr

Classifications

• G06F2221/034

  Test or assess a computer or a system · CPC title

• G06F21/55 Primary

  Detecting local intrusion or implementing counter-measures · CPC title