Dynamic scheduling of tasks for collecting and processing data from external sources

Patent metadata

Field	Value
Publication number	US-8978036-B2
Application number	US-201313953568-A
Country	US
Kind code	B2
Filing date	Jul 29, 2013
Priority date	Jul 29, 2013
Publication date	Mar 10, 2015
Grant date	Mar 10, 2015

Abstract

A scheduler manages execution of a plurality of data-collection jobs, assigns individual jobs to specific forwarders in a set of forwarders, and generates and transmits tokens (e.g., pairs of data-collection tasks and target sources) to assigned forwarders. The forwarder uses the tokens, along with stored information applicable across jobs, to collect data from the target source and forward it onto an indexer for processing. For example, the indexer can then break a data stream into discrete events, extract a timestamp from each event and index (e.g., store) the event based on the timestamp. The scheduler can monitor forwarders' job performance, such that it can use the performance to influence subsequent job assignments. Thus, data-collection jobs can be efficiently assigned to and executed by a group of forwarders, where the group can potentially be diverse and dynamic in size.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
identifying, by a scheduler, a plurality of jobs, wherein each job in the plurality of jobs comprises collecting data from one or more external sources; job includes collecting data from an external source;
transmitting, at a direction of the scheduler, credentials to each forwarder in a set of multiple forwarders, wherein each forwarder stores the credentials;
generating, by the scheduler, a configuration token…

Assignees

• Splunk Inc

Inventors

• Fletcher Tristan
• Bingham Brian

Classifications

• G06F9/5033 Primary

  Physics · mapped topic

• G06F9/5011 Primary

  Physics · mapped topic

• G06F9/505

  Physics · mapped topic

• H04L67/59

  Electricity · mapped topic

• G06F16/316

  Physics · mapped topic