Generating and transforming timestamped event data at a remote capture agent

Patent metadata

Field	Value
Publication number	US-10348583-B2
Application number	US-201715582309-A
Country	US
Kind code	B2
Filing date	Apr 28, 2017
Priority date	Apr 15, 2014
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains, at a remote capture agent, configuration information for the remote capture agent from a configuration server over a network. Next, the system uses the configuration information to configure the generation of event data from network data obtained from network packets at the remote capture agent. The system then uses the configuration information to configure transformation of the event data or the network data into transformed event data at the remote capture agent.

First claim

What is claimed is:

1. A computer-implemented method performed by a remote capture agent coupled to a network, comprising:
obtaining configuration information from a configuration server over a network;
monitoring network traffic comprising a plurality of network packets;
generating, based on the configuration information, timestamped event data from at least one network packet of the plurality of network packets by segmenting the at least one network packet into events and associating timestamps with the events;
transforming, based on the same configuration information, the timestamped event data into transformed event data; and
sending an event stream of timestamped event data including the transformed event data to another component on the network for subsequent processing.

2. The computer-implemented method of claim 1 , wherein the another component on the network includes a transformation server that performs at least one additional transformation to timestamped event data in the event stream.

3. The computer-implemented method of claim 1 , wherein the event stream includes a type of event data specified in the configuration information.

4. The computer-implemented method of claim 1 , further comprising:
receiving an update to the configuration information from the configuration server; and
generating, based on the updated configuration information, timestamped event data from at least one network packet of the plurality of network packets.

5. The computer-implemented method of claim 1 , wherein transformation of the timestamped event data comprises at least one of an aggregation, a calculation, a filter, a normalization, and a formatting.

6. The computer-implemented method of claim 1 , wherein transformation of the timestamped event data comprises:
obtaining a time interval associated with the timestamped event data or the network traffic; and
aggregating the timestamped event data or the network data within the time interval into at least one of an event count, a statistic, and a uniqueness count.

7. The computer-implemented method of claim 1 , wherein the configuration server obtains the configuration information from an application used to access the transformed event data.

8. The computer-implemented method of claim 1 , wherein the configuration information comprises at least one of an identifier, a description, an event stream type, a custom field, and an additional parameter.

9. The computer-implemented method of claim 1 , wherein the remote capture agent is installed in a virtual computing environment.

10. The computer-implemented method of claim 1 , wherein the transformation of the timestamped event data comprises:
identifying a network address in the timestamped event data;
identifying related data in a lookup table based on the identified network address; and
including the related data in the transformed event data.

11. A remote capture agent, comprising:
a processor;
a memory storing instructions which, when executed by the processor, cause the remote capture agent to:
obtain configuration information from a configuration server over a network;
monitor network traffic comprising a plurality of network packets;
generate, based on the configuration information, timestamped event data from at least one network packet of the plurality of network packets by segmenting the at least one network packet into events and associating timestamps with the events;
transform, based on the same configuration information, the timestamped event data into transformed event data; and
send an event stream of timestamped event data including the transformed event data to another component on the network for subsequent processing.

12. The remote capture agent of claim 11 , wherein the another component on the network includes a transformation server that perform at least one additional transformation to timestamped event data in the event stream.

13. The remote capture agent of claim 11 , wherein the event stream includes a type of event data specified in the configuration information.

14. The remote capture agent of claim 11 , wherein transformation of the timestamped event data comprises at least one of an aggregation, a calculation, a filter, a normalization, and a formatting.

15. The remote capture agent of claim 11 , wherein transformation of the timestamped event data comprises:
obtaining a time interval associated with the timestamped event data or the network traffic; and
aggregating the timestamped event data or the network data within the time interval into at least one of an event count, a statistic, and a uniqueness count.

16. The remote capture agent of claim 11 , wherein the configuration server obtains the configuration information from an application used to access the transformed event data.

17. The remote capture agent of claim 11 , wherein the configuration information comprises at least one of an identifier, a description, an event stream type, a custom field, and an additional parameter.

18. The remote capture agent of claim 11 , wherein the transformation of the timestamped event data comprises:
identifying a network address in the timestamped event data;
identifying related data in a lookup table based on the identified network address; and
including the related data in the transformed event data.

19. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause a remote capture agent coupled to a network to perform operations comprising:
obtaining configuration information from a configuration server over a network;
monitoring network traffic comprising a plurality of network packets;
generating, based on the configuration information, timestamped event data from at least one network packet of the plurality of network packets by segmenting the at least one network packet into events and associating timestamps with the events;
transforming, based on the same configuration information, the timestamped event data into transformed event data; and
sending an event stream of timestamped event data including the transformed event data to another component on the network for subsequent processing.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the another component on the network includes a transformation server that performs at least one additional transformation to timestamped event data in the event stream.

21. The non-transitory computer-readable storage medium of claim 19 , wherein the event stream includes a type of event data specified in the configuration information.

22. The non-transitory computer-readable storage medium of claim 19 , wherein transformation of the timestamped event data comprises at least one of an aggregation, a calculation, a filter, a normalization, and a formatting.

23. The non-transitory computer-readable storage medium of claim 19 , wherein transformation of the timestamped event data comprises:
obtaining a time interval associated with the timestamped event data or the network traffic; and
aggregating the timestamped event data or the network data within the time interval into at least one of an event count, a statistic, and a uniqueness count.

24. The non-transitory computer-readable storage medium of claim 19 , wherein the configuration server obtains the configuration information from an application used to access the transformed event data.

25. The

Assignees

• Splunk Inc

Inventors

• Dickey Michael

Classifications

• H04L41/046

  comprising network management agents or mobile agents therefor · CPC title

• H04L41/0816

  the condition being an adaptation, e.g. in response to network events · CPC title

• H04L41/0856 Primary

  by backing up or archiving configuration information · CPC title

• H04L43/106

  using time related information in packets, e.g. by adding timestamps · CPC title

• H04L43/04 Primary

  Processing captured monitoring data, e.g. for logfile generation · CPC title