Protocol-based capture of network data using remote capture agents

Patent metadata

Field	Value
Publication number	US-9838512-B2
Application number	US-201414528898-A
Country	US
Kind code	B2
Filing date	Oct 30, 2014
Priority date	Oct 30, 2014
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.

First claim

What is claimed is:

1. A method performed by a remote capture agent coupled to a network, the method comprising:
monitoring network traffic comprising a plurality of network packets on the network;
identifying a packet flow including at least one network packet from the plurality of network packets, wherein the packet flow is associated with a communication path between a source and a destination;
identifying a protocol associated with the packet flow;
generating, based on configuration information associated with the identified protocol, an event stream from the packet flow, wherein the event stream comprises time-series event data created based on data contained in network packets of the packet flow; and
transmitting the event stream to another component on the network.

2. The method of claim 1 , further comprising:
obtaining, at the remote capture agent, the configuration information from a configuration server over the network; and
using the configuration information to configure the generation of the time-series event data from the network packets during runtime of the remote capture agent.

3. The method of claim 1 , further comprising:
wherein the packet flow is a first packet flow, the identified protocol is a first identified protocol, and the event stream is a first event stream;
identifying a second packet flow including at least one network packet from the plurality of network packets;
identifying a second protocol associated with the second packet flow;
generating, based on configuration information associated with the second identified protocol, a second event stream from the second packet flow at the remote capture agent, wherein the second event stream comprises time-series event data created based on data contained in network packets of the second packet flow; and
transmitting the second event stream to another component on the network.

4. The method of claim 1 , further comprising:
identifying the network packets of the packet flow based on control information in the network packets.

5. The method of claim 1 , further comprising:
assembling the packet flow from the network packets; and
in response to detecting encryption of the network packets of the packet flow, decrypting the network packets in the packet flow prior to obtaining the protocol for the packet flow.

6. The method of claim 1 , wherein the network packets of the packet flow are associated with at least one of:
the source;
the destination;
a network address;
a port; and
a transport layer protocol.

7. The method of claim 1 , wherein generating the event stream from the packet flow further comprises:
identifying one or more event attributes associated with the protocol from the configuration information;
extracting the one or more event attributes from the network packets in the packet flow; and
including the extracted one or more event attributes in the event stream.

8. The method of claim 1 , wherein generating the event stream from the packet flow further comprises:
identifying one or more event attributes associated with the protocol from the configuration information;
extracting the one or more event attributes from the network packets in the packet flow;
transforming, based on the configuration information, the extracted one or more event attributes; and
including the transformed one or more event attributes in the event stream.

9. The method of claim 1 , wherein the protocol comprises at least one of:
a transport layer protocol;
a session layer protocol;
a presentation layer protocol; and
an application layer protocol.

10. A remote capture agent, comprising:
a processor;
a non-transitory computer readable storage medium storing instructions which, when executed by the processor, cause the remote capture agent to:
monitor network traffic on a network, the network traffic comprising a plurality of network packets;
identify a packet flow including at least one network packet from the plurality of network packets, wherein the packet flow is associated with a communication path between a source and a destination;
identify a protocol associated with the packet flow;
generate, based on configuration information associated with the identified protocol, an event stream from the packet flow, wherein the event stream comprises time-series event data created based on data contained in network packets of the packet flow; and
transmit the event stream to another component on the network.

11. The remote capture agent of claim 10 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
obtain the configuration information from a configuration server over a network; and
use the configuration information to configure the generation of the time-series event data from the network packets during runtime of the remote capture agent.

12. The remote capture agent of claim 10 , wherein the packet flow is a first packet flow, the identified protocol is a first identified protocol, the event stream is a first event stream, and the instructions, when executed by the processor, further cause the remote capture agent to:
identify a second packet flow including at least one network packet from the plurality of network packets;
identify a second protocol associated with the second packet flow;
generate, based on configuration information associated with the second identified protocol, a second event stream from the second packet flow at the remote capture agent, wherein the second event stream comprises time-series event data created based on data contained in network packets of the second packet flow; and
transmit the second event stream to another component on the network.

13. The remote capture agent of claim 10 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
identify the network packets of the packet flow based on control information in the network packets;
assemble the packet flow from the network packets; and
in response to detecting encryption of the network packets of the packet flow, decrypt the network packets in the packet flow prior to obtaining the protocol for the packet flow.

14. The remote capture agent of claim 10 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
identify one or more event attributes associated with the protocol from the configuration information;
extract the one or more event attributes from the network packets in the packet flow; and
include the extracted one or more event attributes in the event stream.

15. The remote capture agent of claim 10 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
identify one or more event attributes associated with the protocol from the configuration information;
extract the one or more event attributes from the network packets in the packet flow;
transform, based on the configuration information, the extracted one or more event attributes; and
include the transformed one or more event attributes in the event stream.

16. The remote capture agent of claim 10 , wherein the protocol comprises at least one of:
a transport layer protocol;
a session layer protocol;
a presentation layer protocol; and
an application layer protocol.

17. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform operations comprising:
monitoring network traffic on a network, the network traffic comprising a

Assignees

• Splunk Inc

Inventors

• Shcherbakov Vladimir A
• Dickey Michael R

Classifications

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title

• H04L69/22 Primary

  Parsing or analysis of headers · CPC title