Method and apparatus for grouping features into bins with selected bin boundaries for use in anomaly detection

Patent metadata

Field	Value
Publication number	US-10154053-B2
Application number	US-201615090992-A
Country	US
Kind code	B2
Filing date	Apr 5, 2016
Priority date	Jun 4, 2015
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

In one embodiment, a method includes receiving network data at an analytics device, identifying features for the network data at the analytics device, grouping each of the features into bins of varying width at the analytics device, the bins comprising bin boundaries selected based on a probability that data within each of the bins follows a discrete uniform distribution, and utilizing the binned features for anomaly detection. An apparatus and logic are also disclosed herein.

First claim

What is claimed is:

1. A method comprising:
receiving network data at a processor of an analytics device, the network data collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network;
processing the network data at the processor of the analytics device, wherein processing comprises:
identifying features for the network data;
determining transition points for each of said features in a histogram;
grouping each of said features into bins of varying width in the histogram, wherein said width defines a range of said features in each of said bins;

wherein said transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution; and
inputting said binned features into an algorithm for anomaly detection.

2. The method of claim 1 wherein said bin boundaries are assigned at transition points within the features.

3. The method of claim 1 further comprising updating said bin boundaries upon receiving new network data.

4. The method of claim 1 wherein each of said features comprises univariate data.

5. The method of claim 1 wherein said features comprise numeric features.

6. The method of claim 1 further comprising defining said bin boundaries utilizing a Pearson chi-square test to determine said probability.

7. The method of claim 1 further comprising identifying observation counts for minimum and maximum values of one of said bins for use in testing said probability.

8. The method of claim 1 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

9. The method of claim 1 wherein said selected bin boundaries retain distribution characteristics comprising spikes and areas of sparseness in said binned features.

10. The method of claim 1 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

11. An apparatus comprising:
an interface for receiving network data; and
a processor for identifying features for the network data, determining transition points for each of said features in a histogram, grouping each of said features into bins of varying width in the histogram, and inputting said binned features into an algorithm for anomaly detection;
wherein the transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution and wherein said width defines a range of said features in each of said bins.

12. The apparatus of claim 11 wherein said bin boundaries are assigned at transition points within the features.

13. The apparatus of claim 11 wherein the processor is further configured to define said bin boundaries utilizing a Pearson chi-square test to determine said probability.

14. The apparatus of claim 11 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

15. The apparatus of claim 11 wherein said selected bin boundaries retain distribution characteristics comprising spikes and areas of sparseness in said binned features.

16. The apparatus of claim 11 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

17. Logic encoded on one or more non-transitory computer readable media for execution and when executed operable to:
identify features for network data;
determine transition points for each of said features in a histogram;
group each of said features into bins of varying width in the histogram, wherein said width defines a range of said features in each of said bins; and
input said binned features into an algorithm for anomaly detection;
wherein said transition points define bin boundaries in the histogram, said transition points selected based on a probability that data within each of the bins follows a discrete uniform distribution.

18. The logic of claim 17 further operable to identify observation counts for minimum and maximum values of one of said bins for use in testing said probability.

19. The logic of claim 17 wherein said probability is compared to a predetermined value selected based on a desired granularity of data and available storage space.

20. The logic of claim 17 wherein the network data is collected from a plurality of sensors distributed throughout a network to monitor network flows within the network from multiple perspectives in the network.

Assignees

• Cisco Tech Inc

Inventors

• Yadav Navindra
• Scheib Ellen
• Agasthy Rachita

Classifications

• H04L63/1425 Primary

  Traffic logging, e.g. anomaly detection · CPC title

• H04L41/142 Primary

  using statistical or mathematical methods · CPC title

• H04L63/20

  for managing network security; network security policies in general (filtering policies H04L63/0227) · CPC title

• H04L43/062

  related to network traffic · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title