Online anomaly detection of vector embeddings

Patent metadata

Field	Value
Publication number	US-12250239-B2
Application number	US-202318489097-A
Country	US
Kind code	B2
Filing date	Oct 18, 2023
Priority date	Jan 31, 2020
Publication date	Mar 11, 2025
Grant date	Mar 11, 2025

Abstract

Disclosed herein are system, method, and computer program product embodiments for providing an anomaly detection system. Some aspects of this disclosure include a method for detecting anomaly in a network device. The method includes determining one or more similarity values between a flow vector corresponding to a flow associated with the network device and one or more flow clusters associated with the network device. The method further includes determining a maximum similarity value as a maximum of the one or more similarity values and comparing the maximum similarity value to a threshold. The method also includes, in response to the maximum similarity value being equal to or greater than the threshold, updating a flow cluster associated with the maximum similarity value. The method also includes, in response to the maximum similarity measure being less than the threshold, detecting the anomaly in the network device.

First claim

What is claimed is:

1. A method, comprising:
determining, by at least one processor of an anomaly detection system, a maximum similarity value as a maximum of a plurality of similarity values between a flow vector and a plurality of flow clusters associated with a network device;
comparing, by the at least one processor of the anomaly detection system, the maximum similarity value to a threshold, wherein the threshold is based on a minimum confidence threshold; and
in response to the maximum similarity value being less than the threshold:
detecting an anomaly in the network device;
generating an alert message based on the detected anomaly; and
generating a new flow cluster based on the flow vector, wherein the new flow cluster is stored in a memory for a subsequent anomaly detection.

2. The method of claim 1 , wherein the minimum confidence threshold is specific to one or more of a network including the network device, to the network device, or to a type of a network flow associated with the flow vector.

3. The method of claim 1 , further comprising:
determining the minimum confidence threshold by analyzing one or more of a network including the network device, the network device, or a type of a network flow associated with the flow vector.

4. The method of claim 1 , further comprising:
dynamically adjusting the minimum confidence threshold based on one or more of a network including the network device, the network device, or network data associated with the network.

5. The method of claim 1 , further comprising:
in response to the maximum similarity value being equal to or greater than the threshold, updating, by the at least one processor of the anomaly detection system, a flow cluster associated with the maximum similarity value by combining the flow cluster associated with the maximum similarity value with the flow vector, wherein the updated flow cluster is stored in the memory for the subsequent anomaly detection.

6. The method of claim 5 , wherein the combining the flow cluster associated with the maximum similarity value with the flow vector comprises:
determining an exponentially weighted moving average between the flow vector and the flow cluster associated with the maximum similarity value; and
updating a timestamp associated with the flow cluster associated with the maximum similarity value, wherein the updated timestamp indicates a time that the flow cluster associated with the maximum similarity value is updated.

7. The method of claim 1 , further comprising:
receiving two or more initial flow vectors, wherein the two or more initial flow vectors are based on a behavioral model of the network device generated based on processing a plurality of records associated with the network device, and wherein the two or more initial flow vectors are stored in the memory; and
generating, based on the two or more initial flow vectors, the flow vector corresponding to the network device, wherein the flow vector is stored in the memory.

8. The method of claim 7 , wherein:
the receiving the two or more initial flow vectors further comprises:
receiving a first initial flow vector corresponding to a first flow associated with the network device; and
receiving a second initial flow vector corresponding to a second flow associated with the network device, and
the generating the flow vector further comprises:
determining a similarity value between the first initial flow vector and the second initial flow vector;
comparing the similarity value to a similarity threshold; and
in response to the similarity value being equal to or greater than the similarity threshold, generating the flow vector.

9. The method of claim 8 , wherein the generating the flow vector comprises creating a weighted average of the first initial flow vector and the second initial flow vector.

10. The method of claim 8 , wherein the first and second initial flow vectors are stored in contiguous memory spaces in the memory.

11. The method of claim 1 , further comprising:
dynamically updating the threshold based on at least one of a flow associated with the network device or a behavior of the network device.

12. A method, comprising:
determining, by at least one processor of an anomaly detection system, a plurality of similarity values between a network flow and a plurality of flow clusters associated with a network device;
comparing, by the at least one processor of the anomaly detection system, a maximum similarity value to a threshold, wherein the threshold is based on a minimum confidence threshold;
determining, by the at least one processor of the anomaly detection system and based on the comparing and at a flow level, whether the network flow indicates an anomaly in a behavior of the network device; and
in response to determining that the network flow indicates the anomaly in the behavior of the network device:
generating an alert message based on the anomaly, wherein the alert message comprises at least one or more of information associated with the network device with the anomaly, information associated with the network flow that triggered the anomaly, information about a flow vector, or information associated with a flow cluster associated with the maximum similarity value; and
generating a new flow cluster based on the network flow, wherein the new flow cluster is stored in a memory for a subsequent anomaly detection.

13. The method of claim 12 , wherein the minimum confidence threshold is specific to one or more of a network including the network device, to the network device, or to a type of a network flow associated with the network flow.

14. The method of claim 12 , further comprising:
determining the minimum confidence threshold by analyzing one or more of a network including the network device, the network device, or a type of a network flow associated with the network flow.

15. The method of claim 12 , further comprising:
dynamically adjusting the minimum confidence threshold based on one or more of a network including the network device, the network device, or network data associated with the network.

16. The method of claim 12 , further comprising:
in response to determining that the network flow does not indicate the anomaly in the behavior of the network device, updating one of the plurality of flow clusters by combining the one of the plurality of flow clusters with a flow vector associated with the network flow, wherein the updated one of the plurality of flow clusters is stored in the memory for the subsequent anomaly detection.

17. The method of claim 12 , further comprising:
receiving, by the at least one processor of the anomaly detection system, two or more initial flow vectors, wherein the two or more initial flow vectors are based on a behavioral model of a network device generated based on processing a plurality of records associated with the network device, and wherein the two or more initial flow vectors are stored in the memory; and
generating, by the at least one processor of the anomaly detection system and based on the two or more initial flow vectors, a network flow associated with the network device, wherein the network flow is stored in the memory.

18. The method of claim 12 , further comprising:
dynamically updating the threshold based on at least one of a flow associated with the network device or the behavior of the network device.

19. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
determine a minimum confidence threshold by anal

Assignees

• Extreme Networks Inc

Inventors

• Bernardi Giacomo
• Horgan Donagh
• Haskell Jeffrey W
• Nispel Markus

Classifications

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title

• G06F21/552 Primary

  involving long-term monitoring or reporting · CPC title