Automatic generation of generic file signatures

Patent metadata

Field	Value
Publication number	US-9762593-B1
Application number	US-201414481763-A
Country	US
Kind code	B1
Filing date	Sep 9, 2014
Priority date	Sep 9, 2014
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

Systems and methods to automatically generate signatures used to detect malware are provided. The systems and methods use machine learning techniques, to build an over-trained heuristic model to analyze software, cluster identified patterns, validate the clusters against known reputational metrics, automatically create signatures and, in some examples, deploy such signatures to remote computing devices.

First claim

What is claimed is:

1. A method for automatically generating signatures for detecting malware, comprising:
collecting a set of static attributes from a malware dataset and a goodware dataset;
generating a plurality of decision trees from the set of static attributes, wherein each decision tree in the plurality of decision trees comprises a plurality of terminal nodes;
identifying, for each sample in a known-file dataset, a pattern of terminal nodes to which the sample is mapped by the plurality of decision trees, wherein the pattern of terminal nodes of the sample comprises a representation of a terminal node from each decision tree within the plurality of decision trees to which the sample has been mapped;
generating a cluster of samples comprising samples in the known file dataset that have identical patterns of terminal nodes;
validating the cluster of samples against a reputation value range to determine a purity of the cluster of samples; and
generating, based at least in part on the purity of the cluster of samples, a signature for identifying additional files that are similar to the samples in the cluster of samples.

2. The method for automatically generating signatures for detecting malware according to claim 1 , further comprising:
detecting a malicious file that satisfies the signature; and
performing, in response to detecting the malicious file, a security action on the malicious file.

3. The method for automatically generating signatures for detecting malware according to claim 1 , wherein generating the plurality of decision trees from the set of static attributes comprises over-training the plurality of decision trees without restricting the smallest allowable size of nodes within the plurality of decision trees.

4. The method for automatically generating signatures for detecting malware according to claim 1 , wherein:
the known file dataset comprises a plurality of files known to be malicious;
validating the cluster of samples against the reputation value range to determine the purity of the cluster of samples comprises determining that the cluster of samples is a bad cluster.

5. The method for automatically generating signatures for detecting malware according to claim 1 , wherein:
the known file dataset comprises a plurality of files known to be benign;
validating the cluster of samples against the reputation value range to determine the purity of the cluster of samples comprises determining that the cluster of samples is a good cluster.

6. The method for automatically generating signatures for detecting malware according to claim 1 , wherein:
the known file dataset comprises a plurality of files known to be malicious;
validating the cluster of samples against the reputation value range to determine the purity of the cluster of samples comprises determining that the cluster of samples is a suspected bad cluster.

7. The method for automatically generating signatures for detecting malware according to claim 1 , wherein:
the known file dataset comprises a plurality of files known to be benign;
validating the cluster of samples against the reputation value range to determine the purity of the cluster of samples comprises determining that the cluster of samples is a suspected good cluster.

8. A system to automatically generate signatures used to detect malware, comprising:
an attribute collection module, stored in memory, that collects a set of static attributes from a malware dataset and a goodware dataset;
a heuristic module, stored in memory, that generates a plurality of decision trees from the set of static attributes, wherein each decision tree in the plurality of decision trees comprises a plurality of terminal nodes;
a clustering module, stored in memory, that:
identifies, for each sample in a known-file dataset, a pattern of terminal nodes to which the sample is mapped by the plurality of decision trees, wherein the pattern of terminal nodes of the sample comprises a representation of a terminal node from each decision tree within the plurality of decision trees to which the sample has been mapped; and
generates a cluster of samples comprising samples in the known file dataset that have identical patterns of terminal nodes;

a cluster validation module, stored in memory, that validates the cluster of samples against a reputation value range to determine a purity of the cluster of samples;
a signature creation module, stored in memory, that creates, based at least in part on the purity of the cluster of samples, a signature for identifying additional files that are similar to the samples in the cluster of samples; and
at least one physical processor that executes the attribute collection module, the heuristic module, the clustering module, the cluster validation module, and the signature creation module.

9. The system according to claim 8 , further comprising a security module that:
detects a malicious file that satisfies the signature; and
performs, in response to detecting the malicious file, a security action on the malicious file.

10. The system according to claim 8 , wherein the heuristic module generates the plurality of decision trees from the set of static attributes by over-training the plurality of decision trees without restricting the smallest allowable size of nodes within the plurality of decision trees.

11. The system according to claim 8 , wherein:
the known file dataset comprises a plurality of files known to be malicious;
the cluster validation module validates the cluster of samples against the reputation value range to determine the purity of the cluster of samples by determining that the cluster of samples is a bad cluster.

12. The system according to claim 8 , wherein:
the known file dataset comprises a plurality of files known to be benign;
the cluster validation module validates the cluster of samples against the reputation value range to determine the purity of the cluster of samples by determining that the cluster of samples is a good cluster.

13. The system according to claim 8 , wherein:
the known file dataset comprises a plurality of files known to be malicious;
the cluster validation module validates the cluster of samples against the reputation value range to determine the purity of the cluster of samples by determining that the cluster of samples is a suspected bad cluster.

14. The system according to claim 8 , wherein:
the known file dataset comprises a plurality of files known to be benign;
the cluster validation module validates the cluster of samples against the reputation value range to determine the purity of the cluster of samples by determining that the cluster of samples is a suspected good cluster.

15. A non-transitory computer-readable medium comprising computer executable instructions that when executed by at least one processor of a computing device, cause the computing device to:
collect a set of static attributes from a malware dataset and a goodware dataset;
generate a plurality of decision trees from the set of static attributes, wherein each decision tree in the plurality of decision trees comprises a plurality of terminal nodes;
identify, for each sample in a known-file dataset, a pattern of terminal nodes to which the sample is mapped by the plurality of decision trees, wherein the patter of terminal nodes of the sample comprises a representation of a terminal node from each decision tree within the plurality of decision trees to which the sample has been mapped;
generate a cluster of samples comprising samples in the known file dataset that have identical patterns of terminal nodes;
valida

Assignees

• Symantec Corp

Inventors

• Kennedy Mark
• Coleman Kenneth

Classifications

• H04L63/1416 Primary

  Event detection, e.g. attack signature detection · CPC title

• H04L63/164

  at the network layer · CPC title

• H04L63/145

  the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms · CPC title

• G06F21/64

  Protecting data integrity, e.g. using checksums, certificates or signatures · CPC title

• G06F21/56

  Computer malware detection or handling, e.g. anti-virus arrangements · CPC title