Malware data clustering

Patent metadata

Field	Value
Publication number	US-12238136-B2
Application number	US-202318504392-A
Country	US
Kind code	B2
Filing date	Nov 8, 2023
Priority date	Mar 15, 2013
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

In various embodiments, systems, methods, and techniques are disclosed for generating a collection of clusters of related data from a seed. Seeds may be generated based on seed generation strategies or rules. Clusters may be generated by, for example, retrieving a seed, adding the seed to a first cluster, retrieving a clustering strategy or rules, and adding related data and/or data entities to the cluster based on the clustering strategy. Various cluster scores may be generated based on attributes of data in a given cluster. Further, cluster metascores may be generated based on various cluster scores associated with a cluster. Clusters may be ranked based on cluster metascores. Various embodiments may enable an analyst to discover various insights related to data clusters, and may be applicable to various tasks including, for example, tax fraud detection, beaconing malware detection, malware user-agent detection, and/or activity trend detection, among various others.

First claim

What is claimed is:

1. A computer system comprising:
one or more computer readable storage devices configured to store a plurality of captured communications; and
one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute computer executable instructions to cause the computer system to:
execute a cluster engine configured to at least:
generate, based on a plurality of captured communications, a filtered collection of captured communications, wherein the captured communications include user-agent strings;
determine, based on the filtered collection of captured communications, a first set of captured communications associated with a first time period, and a second set of captured communications associated with a second time period;
identify a first captured communication in the first set of captured communications that is not included among the second set of captured communications, wherein the first captured communication indicates a new user-agent string associated with the first time period and not associated with the second time period;
designate the new user-agent string as a seed;
generate a data item cluster based on the designated seed; and
determine scores for the data item cluster and a plurality of additional data items clusters generated based on user-agent-related data items; and

execute a workflow engine configured to at least:
cause presentation of the data item cluster and the plurality of additional data item clusters in a user interface of a client computing device; and
cause ordering of the presented data item cluster and the plurality of additional data item clusters in the user interface based at least in part on the respective determined scores for the data item cluster and the plurality of additional data item clusters.

2. The computer system of claim 1 , wherein generating the filtered collection of captured communications includes:
removing captured communications with destinations on an approved list of destinations, wherein the approved list of destinations indicates destinations that are unlikely to be related to malware activity.

3. The computer system of claim 2 , wherein generating the filtered collection of captured communications further includes:
removing captured communications with respective user-agent strings that are on an approved list of user-agent strings, wherein the approved list of user-agent strings indicates destinations that are unlikely to be related to malware activity.

4. The computer system of claim 3 , wherein generating the filtered collection of captured communications further includes:
removing captured communications associated with a particular external computer system, wherein the particular external computer system is unlikely to be related to malware activity.

5. The computer system of claim 4 , wherein generating the filtered collection of captured communications further includes:
removing captured communications associated with random user-agent strings.

6. The computer system of claim 5 , wherein the one or more hardware computer processors are configured to execute the computer executable instructions to further cause the computer system to:
identify a quantity of appearances of the new user-agent string in corresponding captured communications among the first set of captured communications; and
determine the quantity is below a predetermined threshold.

7. The computer system of claim 1 , wherein generating the filtered collection of captured communications includes:
removing captured communications with respective user-agent strings that are on an approved list of user-agent strings, wherein the approved list of user-agent strings indicates destinations that are unlikely to be related to malware activity.

8. The computer system of claim 7 , wherein generating the filtered collection of captured communications further includes:
removing captured communications associated with a particular external computer system, wherein the particular external computer system is unlikely to be related to malware activity.

9. The computer system of claim 8 , wherein generating the filtered collection of captured communications further includes:
removing captured communications associated with random user-agent strings.

10. The computer system of claim 1 , wherein generating the filtered collection of captured communications includes:
removing captured communications associated with a particular external computer system, wherein the particular external computer system is unlikely to be related to malware activity.

11. The computer system of claim 10 , wherein generating the filtered collection of captured communications further includes:
removing captured communications associated with random user-agent strings.

12. The computer system of claim 1 , wherein generating the filtered collection of captured communications includes:
removing captured communications associated with random user-agent strings.

13. The computer system of claim 1 , wherein the one or more hardware computer processors are configured to execute the computer executable instructions to further cause the computer system to:
identify a quantity of appearances of the new user-agent string in corresponding captured communications among the first set of captured communications; and
determine the quantity is below a predetermined threshold.

14. The computer system of claim 1 , wherein generating the data item cluster comprises:
adding the seed to the data item cluster; and
adding to the data item cluster one or more user-agent-related data items determined to be associated with the seed, wherein the one or more user-agent-related data items comprise information associated with a computing device.

15. The computer system of claim 14 , wherein the one or more user-agent-related data items further include at least one of: a user of a particular computing device, an internal Internet Protocol address, an external Internet Protocol address, an external domain, an internal computing device, an external computing device, or a host-based event.

16. The computer system of claim 14 , wherein the one or more hardware computer processors are configured to execute the computer executable instructions to further cause the computer system to:
identify the one or more user-agent-related data items based at least on a clustering strategy, wherein the clustering strategy queries one or more cluster data sources to determine at least one of: originating host or destination computing devices associated with the seed, users of originating host computing devices, intrusion prevention system alerts associated with originating host computing devices, internal Internet Protocol addresses associated with originating host computing devices, external Internet Protocol addresses associated with destination computing devices, or external domains associated with the first captured communication.

17. A computer-implemented method comprising:
by one or more hardware processors executing program instructions:
executing a cluster engine configured to perform operations including at least:
generating, based on a plurality of captured communications, a filtered collection of captured communications, wherein the captured communications include user-agent strings;
determining, based on the filtered collection of captured communications, a first set of captured communications associated with a first time period, and a second set of captured communications associated with a second

Assignees

• Palantir Technologies Inc

Inventors

• Singh Harkirat
• Stowe Geoffrey
• Bach Stefan
• Sprague Matthew
• Kross Michael
• Borochoff Adam
• Menon Parvathy
• Harris Michael

Classifications

• G06F16/283

  Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP · CPC title

• G06Q20/382

  insuring higher security of transaction · CPC title

• G06Q40/123

  Tax preparation or submission · CPC title

• G06Q30/0185

  Product, service or business identity fraud · CPC title

• G06Q20/4016

  involving fraud or risk level assessment in transaction processing · CPC title