Clustering approach for detecting DDoS botnets on the cloud from IPFix data

Patent metadata

Field	Value
Publication number	US-10129295-B2
Application number	US-201615253586-A
Country	US
Kind code	B2
Filing date	Aug 31, 2016
Priority date	Aug 31, 2016
Publication date	Nov 13, 2018
Grant date	Nov 13, 2018

Abstract

Use machine learning to train a classifier to classify entities to increase confidence with respect to an entity being part of a distributed denial of service attack. The method includes training a classifier to use a first classification method, to identify probabilities that entities from a set of entities are performing denial of service attacks. The method further includes identifying a subset of entities meeting a threshold probability of performing a denial of service attack. The method further includes using a second classification method, identifying similarity of entities in the subset of entities. The method further includes based on the similarity, classifying individual entities.

First claim

What is claimed is:

1. A system configured to train and use a classifier to classify entities to determine whether the entities are part of a distributed denial of service (DDoS) attack, the system comprising:
one or more hardware processors; and
one or more computer-readable storage devices having stored thereon instructions that are executable by the one or more hardware processors to configure the system to perform at least the following:
train a classifier to use a first classification method to identify probabilities that entities are performing denial of service attacks, the training comprising applying a captured dataset including data flow protocol information associated with known DDoS attacks;
using the trained classifier, identify a subset of entities from a set of candidate entities that meet or exceed a threshold probability of performing a denial of service attack;
using a second classification method, identify similarity of entities in the identified subset of entities; and
based on the similarity, classify individual entities of the subset of entities as belonging to one or more similarity subgroups, each similarity subgroup comprising entities having a probability of participating in a same DDoS.

2. The system of claim 1 , wherein the second classification method clusters similar entities into similarity clusters.

3. The system of claim 2 , wherein the one or more computer-readable storage devices further have stored thereon instructions that are executable by the one or more hardware processors to configure the computer system to identify a cluster as a set of compromised entities.

4. The system of claim 1 , wherein classifying individual entities comprises identifying entities as performing denial of service.

5. The system of claim 1 , wherein the one or more computer-readable storage devices further have stored thereon instructions that are executable by the one or more hardware processors to configure the computer system to identify entities in a particular botnet based on similarity.

6. The system of claim 1 , wherein the one or more computer-readable storage devices further have stored thereon instructions that are executable by the one or more processors to configure the computer system to identify entities infected by the same means based on similarity.

7. The system of claim 6 , wherein the same means comprises the same malicious software.

8. The system of claim 6 , wherein the same means comprises the same command and control.

9. The system of claim 1 , wherein using the second classification method, to identify similarity of entities in the subset of entities comprises using the L-method.

10. The system of claim 1 , wherein using the second classification method, to identify similarity of entities in the subset of entities comprises using hierarchal clustering.

11. The system of claim 1 , wherein the one or more computer-readable storage devices further have stored thereon instructions that are executable by the one or more hardware processors to configure the computer system to correlate entity activity and wherein using hierarchal clustering is based on correlated entity activity.

12. The system of claim 1 , wherein the one or more computer-readable storage devices further have stored thereon instructions that are executable by the one or more hardware processors to configure the computer system to use available external data to identify a particular botnet.

13. A computer implemented method for training a classifier for classifying entities to determine whether the entities are part of a distributed denial of service (DDoS) attack, the method comprising:
training a classifier to use a first classification method to identify probabilities that entities are performing denial of service attacks, the training comprising applying a captured dataset including data flow protocol information associated with known DDoS attacks;
using the trained classifier, identifying a subset of entities from a set of candidate entities that meet or exceed a threshold probability of performing a denial of service attack;
using a second classification method, identifying similarity of entities in the subset of identified entities; and
based on the similarity, classifying individual entities of the subset of entities as belonging to one or more similarity subgroups, each similarity subgroup comprising entities having a probability of participating in a same DDoS.

14. The method of claim 13 , wherein the second classification method clusters similar entities into similarity clusters.

15. The method of claim 14 , further comprising identifying a cluster as a set of compromised entities.

16. The method of claim 13 , wherein classifying individual entities comprises identifying entities as performing denial of service.

17. The method of claim 13 , further comprising identifying entities in a particular botnet based on similarity.

18. The method of claim 13 , further comprising identifying entities infected by the same means based on similarity.

19. The method of claim 18 , wherein the same means comprises the same malicious software.

20. A computer system configured to use a trained classifier to classify entities to determine whether the entities are part of a distributed denial of service (DDoS) attack, the system comprising:
a botnet classifier coupled to a plurality of computing entities, the botnet classifier comprising one or more computer processors, wherein the botnet classifier is configured to:
capture data flow protocol information from the entities in the plurality of entities;
provide the captured data flow protocol information from the entities to a trained classifier, the trained classifier having been trained by applying previously captured data including data flow protocol information associated with known DDoS attacks;
the trained classifier implementing a first classification method to identify probabilities that entities are performing denial of service attacks based on the captured data flow protocol information;
identify a subset of entities from a set of candidate entities that meet or exceed a threshold probability of performing a denial of service attack;
use a second classification method, identify similarity of entities in the identified subset of entities; and
based on the similarity, classify individual entities of the subset of entities as belonging to one or more similarity subgroups, each similarity subgroup comprising entities having a probability of participating in a same DDoS.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Karin Omer
• Ronen Royi
• Neuvirth Hani
• Vilnai Roey

Classifications

• H04L63/1425

  Traffic logging, e.g. anomaly detection · CPC title

• G06N20/00

  Machine learning · CPC title

• H04L63/1458 Primary

  Denial of Service · CPC title

• G06F16/35

  Clustering; Classification · CPC title

• H04L63/1466

  Active attacks involving interception, injection, modification, spoofing of data unit addresses, e.g. hijacking, packet injection or TCP sequence number attacks · CPC title