Dynamically recalibrating machine learning model parameters

Patent metadata

Field	Value
Publication number	US-12580950-B2
Application number	US-202418752490-A
Country	US
Kind code	B2
Filing date	Jun 24, 2024
Priority date	Jun 24, 2024
Publication date	Mar 17, 2026
Grant date	Mar 17, 2026

Abstract

Discussed herein are methods and systems for dynamically recalibrating machine learning model parameters. In one method, a server executes one or more prediction models to process network operations from various data feeds, in order to identify the likelihood of these operations being fraudulent or malicious. The server monitors performance data, such as the operation and execution metrics of network operations, and evaluates whether the performance values, like recall values, meet defined thresholds. If the performance data fails to meet these thresholds, the server employs a function-generation machine learning model to predict a threshold modification function. This modification function is then applied to adjust the relevant thresholds. Utilizing the modification function, the server dynamically revises one or more parameters of the prediction models to enhance their accuracy and efficacy.

First claim

What is claimed is:

1 . A system for enabling dynamic response of pre-existing models via dynamic modification of model thresholds in connection with performance drops, the system comprising:
one or more processors and memory storing instructions that, when executed by the one or more processors, cause operations comprising:
execute a malicious activity prediction model, an operation failure prediction model, and a testing activity prediction model for processing network operations obtained via one or more data feeds,
wherein the malicious activity prediction model is configured to determine a first likelihood that a network operation corresponds to malicious activity and generate a malicious activity indication when the first likelihood satisfies a first threshold,
wherein the operation failure prediction model is configured to determine a second likelihood that the network operation corresponds to a failure and generate a failure indication when the second likelihood satisfies a second threshold, and
wherein the testing activity prediction model is configured to determine a third likelihood that the network operation corresponds to a testing activity and generate a testing indication when the third likelihood satisfies a third threshold; and

in response to determining that a recall value for a computing infrastructure of a set of computing infrastructures fails to satisfy a recall threshold associated with the computing infrastructure in connection with processing the network operations:
execute a function-generation machine learning model to predict a threshold modification function to be applied to the first, second, or third threshold; and
in connection with processing a first network operation for the computing infrastructure via the malicious activity prediction model, the operation failure prediction model, and the testing activity prediction model, dynamically revise the first, second, or third threshold in accordance with the threshold modification function.

2 . The system of claim 1 , wherein the one or more processors are further configured to:
train the function-generation machine learning model to predict the threshold modification function using a training dataset comprising historical network operations associated with at least one computing infrastructure.

3 . The system of claim 1 , wherein the function-generation machine learning model is trained to predict the threshold modification function corresponding to a timestamp associated with the network operation.

4 . The system of claim 1 , wherein the function-generation machine learning model is trained to predict the threshold modification function corresponding to one or more network operations clustered in accordance with a likelihood of similarity in at least one attribute.

5 . The system of claim 1 , wherein the one or more processors are configured to:
execute the function-generation machine learning model to predict the threshold modification function to be applied to the first, second, or third threshold; and dynamically revise the first, second, or third threshold of the malicious activity prediction, operation failure prediction, or testing activity prediction model in accordance with the threshold modification function in connection with processing the first network operation for the computing infrastructure via the malicious activity prediction, operation failure prediction model, and testing activity prediction models.

6 . The system of claim 1 , wherein the malicious activity prediction model is configured to determine the first likelihood of a first one of the network operation corresponding to the malicious activity, the failure, or the testing activity, the operation failure prediction model is configured to determine the second likelihood of a different one of the network operation corresponding to the malicious activity, the failure, or the testing activity, and the testing activity prediction model is configured to determine the third likelihood of a different one of the network operation corresponding to the malicious activity, the failure, or the testing activity.

7 . A method comprising:
executing, by one or more processors, a first prediction model and a second prediction model for processing network operations obtained via one or more data feeds,
wherein the first prediction model is configured to determine a first likelihood that a network operation corresponds to a first label and generate an indication of the first label when the first likelihood satisfies a first threshold, and the second prediction model is configured to determine a second likelihood that the network operation corresponds to a second label and generate an indication of the second label when the second likelihood satisfies a second threshold;
in connection with processing the network operations, determining, by one or more processors, that performance data for a computing infrastructure of a set of computing infrastructures fails to satisfy a performance threshold associated with the computing infrastructure; and in response to a determination that the performance data fails to satisfy the performance threshold associated with the computing infrastructure:
executing, one or more processors, a machine learning model to predict a threshold modification to be applied to the first or second threshold; and
in connection with processing a first network operation for the computing infrastructure via the first and second models, dynamically revising, by one or more processors, the first or second threshold of the first or second prediction model in accordance with the threshold modification.

8 . The method of claim 7 , further comprising:
training, by one or more processors, the machine learning model to predict the threshold modification function using a training dataset comprising historical network operations associated with at least one computing infrastructure.

9 . The method of claim 7 , wherein the machine learning model is trained to predict the threshold modification corresponding to a timestamp associated with the network operation.

10 . The method of claim 7 , wherein the machine learning model is trained to predict the threshold modification corresponding to one or more network operations clustered in accordance with a likelihood of similarity in at least one attribute.

11 . The method of claim 7 , wherein the machine learning model is trained to predict the threshold modification corresponding to an operational parameter of the computing infrastructure.

12 . The method of claim 7 , wherein:
executing the machine learning model to predict the threshold modification comprises executing the machine learning model to predict a threshold modification function to be applied to the first or second threshold; and dynamically revising the first or second threshold of the first or second prediction model comprises dynamically revising the first or second threshold of the first or second prediction model in accordance with the threshold modification function in connection with processing the first network operation for the computing infrastructure via the first and second models.

13 . The method of claim 7 , wherein the first model is configured to determine the first likelihood of a first one of the network operation corresponding to malicious activity, failure, or the network operation corresponding to a testing activity, and wherein the second model is configured to determine the second likelihood of a different one of the network operation corresponding to malicious activity, failure, or the network operation corresponding to a testing activity.

Assignees

• Stripe Inc
• Stripe Llc

Inventors

• Hegde Chiranth Manjunath
• Lin Michael Joseph
• Li Yafu
• Kontaxis Andrew
• Bernardo Maria Jose Gonzalez

Classifications

• H04W24/02

  Arrangements for optimising operational condition · CPC title

• H04L63/1441 Primary

  Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002) · CPC title