Content delivery network (CDN) bot detection using primitive and compound feature sets

Patent metadata

Field	Value
Publication number	US-11290468-B2
Application number	US-202016922009-A
Country	US
Kind code	B2
Filing date	Jul 7, 2020
Priority date	Apr 16, 2018
Publication date	Mar 29, 2022
Grant date	Mar 29, 2022

Abstract

A method of detecting bots, preferably in an operating environment supported by a content delivery network (CDN) that comprises a shared infrastructure of distributed edge servers from which CDN customer content is delivered to requesting end users (clients). The method begins as clients interact with the edge servers. As such interactions occur, transaction data is collected. The transaction data is mined against a set of “primitive” or “compound” features sets to generate a database of information. In particular, preferably the database comprises one or more data structures, wherein a given data structure associates a feature value with its relative percentage occurrence across the collected transaction data. Thereafter, and upon receipt of a new transaction request, primitive or compound feature set data derived from the new transaction request are compared against the database. Based on the comparison, an end user client associated with the new transaction request is then characterized, e.g., as being associated with a human user, or a bot.

First claim

What we claim follows below:

1. A method of bot detection in a content delivery network (CDN) comprising a distributed set of edge servers, comprising:
as clients interact with the edge servers, collecting transaction data;
mining transaction data against a set of primitive or compound features sets to generate a database, wherein the feature sets can comprise any one of: a canvas fingerprint, one or more browser parameters, device size, user agent, TLS fingerprint, request headers, and one or more additional features, and combinations thereof, the database comprising one or more data structures, wherein a given data structure associates a feature value with its relative percentage occurrence across the collected transaction data; and
upon receipt of a new transaction request, comparing primitive or compound feature set data derived from the new transaction request against the database; and
based on the comparison, characterizing an end user client associated with the new transaction request.

2. The method as described in claim 1 wherein a compound feature set is a concatenation of two or more primitive feature sets.

3. The method as described in claim 1 wherein the transaction data is mined periodically.

4. The method as described in claim 1 wherein the end user client is characterized as a bot when the primitive or compound feature set derived from the new transaction request is not found in the database.

5. The method as described in claim 1 wherein the end user client is characterized as a bot when the primitive or compound feature derived from the new transaction request is found in the database but a frequency of occurrence associated with additional transactions associated with the end user client is inconsistent with the relative percentages set forth in the database.

6. The method as described in claim 1 wherein the transaction data is collected across all of the edge servers.

7. The method as described in claim 1 wherein the transaction data is collected across a subset of the edge servers.

8. The method as described in claim 1 further including providing a script for execution on the end user client, wherein the feature set data is collected upon execution of the script.

9. The method as described in claim 1 wherein a compound features set is defined for a pair of primitive feature sets that are shown to be correlated with one another.

10. The method as described in claim 1 wherein the method is carried out in an automated manner.

11. Apparatus, comprising:
a processor;
computer memory holding computer program instructions configured to:
receive a request from a client; compare primitive or compound feature set data derived from the request against a database, wherein the feature sets can comprise any one of: a canvas fingerprint, one or more browser parameters, device size, user agent, TLS fingerprint, request headers, and one or more additional features, and combinations thereof, the database comprising a database of one or more data structures, wherein a given data structure in the database associates a feature value with its relative percentage occurrence across a set of transaction data collected from prior transactions;
based on the comparison, determine whether the client is a bot; and
upon a determination that the client is a bot, take a mitigation action with respect to the request.

12. The apparatus as described in claim 11 wherein the computer program instructions are further configured to characterize the client as a bot when the primitive or compound feature derived from the request is found in the database but a frequency of occurrence associated with additional transactions associated with the client is inconsistent with the relative percentages set forth in the database.

13. The apparatus as described in claim 11 wherein the computer program instructions comprise a content delivery network edge server process.

14. The apparatus as described in claim 11 wherein the compound feature set is a concatenation of two or more primitive feature sets.

15. The apparatus as described in claim 11 wherein the transaction data is mined periodically.

16. An article, comprising program media holding machine-readable code executable by a processor, the machine-readable code comprising code configured to:
collect transaction data;
mine transaction data against a set of primitive or compound features sets to generate a database, wherein the feature sets can comprise any one of: a canvas fingerprint, one or more browser parameters, device size, user agent, TLS fingerprint, request headers, and one or more additional features, and combinations thereof, the database comprising one or more data structures, wherein a given data structure associates a feature value with its relative percentage occurrence across the collected transaction data;
upon receipt of a new transaction request, compare primitive or compound feature set data derived from the new transaction request against the database; and
based on the comparison, characterize an end user client associated with the new transaction request.

17. The article as described in claim 16 wherein the compound feature set is a concatenation of two or more primitive feature sets.

Assignees

• Akamai Tech Inc

Inventors

• Modalavalasa Venkata Sai Kishore
• Kurupati Sreenath
• Vuong Tu

Classifications

• H04N21/2396

  characterized by admission policies (admission control, resource allocation in open networks H04L12/5692; arrangements for network security using user profiles for access control H04L63/102; access security in wireless networks H04W12/08) · CPC title

• H04L63/1416 Primary

  Event detection, e.g. attack signature detection · CPC title

• H04N21/2408

  Monitoring of the upstream path of the transmission network, e.g. client requests (monitoring data switching networks utilization H04L43/0876; scheduling or organising the servicing of application requests H04L67/60) · CPC title

• H04L63/1408 Primary

  by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title

• G06F21/552

  involving long-term monitoring or reporting · CPC title