Method and apparatus to throttle media access by web crawlers
US-9887933-B2 · Feb 6, 2018 · US
US12335317B2 · US · B2
| Field | Value |
|---|---|
| Publication number | US-12335317-B2 |
| Application number | US-202418593909-A |
| Country | US |
| Kind code | B2 |
| Filing date | Mar 2, 2024 |
| Priority date | Oct 28, 2015 |
| Publication date | Jun 17, 2025 |
| Grant date | Jun 17, 2025 |
A practical reading order for non-experts. Skip the full description unless you need deep technical detail.
What the patent document calls the invention.
A short plain-language summary of the technical disclosure.
Who owns or filed the patent and who is credited as inventor.
Filing, priority, publication, and grant dates set the timeline.
The legal scope of protection — read this for what is actually claimed.
Technology tags used to group this patent with similar filings.
Prior art links and similar publications in this corpus.
Official abstract text for this publication.
A system and method for cybersecurity reconnaissance, analysis, and scoring that uses distributed, cloud-based computing services to provide sufficient scalability for analysis of enterprise IT networks using only publicly available characterizations. The system and method comprise an in-memory associative array which manages a queue of vulnerability search tasks through a public-facing proxy network. The public-facing proxy network has search nodes configurable to present the network to search tools in a desired manner to control certain aspects of the search to obtain the desired results. A distributed data processing engine and cloud-based storage are used to provide scalable computing power and storage. Each of the cloud-based computing services is containerized and orchestrated for management and efficient scaling purposes.
Opening claim text (preview).
What is claimed is: 1. A computing system for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, the computing system comprising: one or more hardware processor configured for: upon request by a user application: receiving a domain name for reconnaissance and scoring; and creating a first queue of Internet search tasks for the domain name using an in-memory associative array service, the search tasks comprising searches for, and receipt of search results for: a domain name system search for domain name system records; a domain name system search for domain name system sender policy framework records; a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and a domain name system search for domain name system zone transfer records; implementing the first queue of Internet search tasks through one or more selectable attribute nodes of a public-facing proxy network; identifying Internet protocol addresses associated with the domain name from the domain name system records; and implementing a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each of a list of open ports for the domain name; receiving a cybersecurity scoring model comprising category weights for a plurality of categories drawn from: domain name system records, domain name system sender policy framework records, domain name system domain-based message authentication, reporting, and conformance records, zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights; retrieving the search results and the list of open ports stored in a cloud-based storage bin; calculating a cybersecurity score by applying the algorithm to the weighted categories; and generating a cybersecurity profile for the domain name based on the cybersecurity score. 2. A computer-implemented method for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, the computer-implemented method comprising: upon request by a user application: receiving a domain name for reconnaissance and scoring; and creating a first queue of Internet search tasks for the domain name using an in-memory associative array service, the search tasks comprising searches for, and receipt of search results for: a domain name system search for domain name system records; a domain name system search for domain name system sender policy framework records; a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and a domain name system search for domain name system zone transfer records; implementing the first queue of Internet search tasks through one or more selectable attribute nodes of a public-facing proxy network; identifying Internet protocol addresses associated with the domain name from the domain name system records; and implementing a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each of a list of open ports for the domain name; receiving a cybersecurity scoring model comprising category weights for a plurality of categories drawn from: domain name system records, domain name system sender policy framework records, domain name system domain-based message authentication, reporting, and conformance records, zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights; retrieving the search results and the list of open ports stored in a cloud-based storage bin; calculating a cybersecurity score by applying the algorithm to the weighted categories; and generating a cybersecurity profile for the domain name based on the cybersecurity score. 3. A system for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, comprising one or more computers with executable instructions that, when executed, cause the system to: upon request by a user application: receive a domain name for reconnaissance and scoring; and create a first queue of Internet search tasks for the domain name using an in-memory associative array service, the search tasks comprising searches for, and receipt of search results for: a domain name system search for domain name system records; a domain name system search for domain name system sender policy framework records; a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and a domain name system search for domain name system zone transfer records; implement the first queue of Internet search tasks through one or more selectable attribute nodes of a public-facing proxy network; identify Internet protocol addresses associated with the domain name from the domain name system records; and implement a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each of a list of open ports for the domain name; receive a cybersecurity scoring model comprising category weights for a plurality of categories drawn from: domain name system records, domain name system sender policy framework records, domain name system domain-based message authentication, reporting, and conformance records, zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights; retrieving the search results and the list of open ports stored in a cloud-based storage bin; calculating a cybersecurity score by applying the algorithm to the weighted categories; and generating a cybersecurity profile for the domain name based on the cybersecurity score. 4. Non-transitory, computer-readable storage media having computer executable instructions embodied thereon that, when executed by one or more processors of a computing system for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, cause the computing system to: upon request by a user application: receive a domain name for reconnaissance and scoring; and create a first queue of Internet search tasks for the domain name using an in-memory associative array service, the search tasks comprising searches for, and receipt of search results for: a domain name system search for domain name system records; a domain name system search for domain name system sender policy framework records; a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and a domain name system search for domain name system zone transfer records; implement the first queue of Internet search tasks through one or more selectable attribute nodes of a public-facing proxy network; identify Internet protocol addresses associated with the domain name from the domain name system records; and implement a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each of a list of open ports for the domain name; receive a cybersecurity scoring model comprising category weights for a plurality of categories drawn from: domain name system records, domain name system sender policy framework records, domain name system domain-based message authentication, reporting, and conformance records, zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights;
Vulnerability analysis · CPC title
by monitoring network traffic (monitoring network traffic per se H04L43/00) · CPC title
Indexing; Web crawling techniques · CPC title
Temporal data queries · CPC title
Countermeasures against malicious traffic (countermeasures against attacks on cryptographic mechanisms H04L9/002) · CPC title
Related publications grouped by family.
Answers are generated from the same data shown on this page.