Configuring generation of multiple event streams from a packet flow

Patent metadata

Field	Value
Publication number	US-10264106-B2
Application number	US-201715799158-A
Country	US
Kind code	B2
Filing date	Oct 31, 2017
Priority date	Oct 30, 2014
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.

First claim

What is claimed is:

1. A computer-implemented method performed by a remote capture agent coupled to a network, the method comprising:
monitoring network traffic comprising a plurality of network packets on the network;
identifying a packet flow in the plurality of network packets, wherein the packet flow is associated with a communication path between a source and a destination;
identifying a first protocol and a second protocol associated with the packet flow;
generating, based on configuration information associated with the first protocol and the second protocol, a first event stream from the packet flow, wherein the first event stream comprises time-series event data created based on first data derived from network packets of the packet flow, and a second event stream from the packet flow, wherein the second event stream comprises time-series event data created based on second data derived from network packets of the packet flow; and
sending the first event stream and the second event stream to another component on the network.

2. The computer-implemented method of claim 1 , further comprising:
obtaining, at the remote capture agent, the configuration information from a configuration server over the network; and
using the configuration information to configure the generation of the first event stream and the second event stream during runtime of the remote capture agent.

3. The computer-implemented method of claim 1 , further comprising:
wherein the packet flow is a first packet flow and the plurality of network packets is a first plurality of network packets;
identifying a second packet flow including a second plurality of network packets;
identifying a third protocol associated with the second packet flow;
generating, based on configuration information associated with the third protocol, a third event stream from the second packet flow at the remote capture agent, wherein the third event stream comprises time-series event data created based on data derived from network packets of the second packet flow; and
transmitting the third event stream to another component on the network.

4. The computer-implemented method of claim 1 , further comprising identifying the plurality of network packets of the packet flow based on control information in the network packets.

5. The computer-implemented method of claim 1 , further comprising:
assembling the packet flow from the plurality of network packets; and
in response to detecting encryption of the network packets of the packet flow, decrypting the network packets in the packet flow prior to identifying the first protocol and the second protocol associated with the packet flow.

6. The computer-implemented method of claim 1 , wherein the network packets of the packet flow are associated with at least one of: the source, the destination, a network address, a port, and a transport layer protocol.

7. The computer-implemented method of claim 1 , wherein generating the first event stream from the packet flow further comprises:
identifying one or more event attributes associated with the first protocol from the configuration information;
extracting the one or more event attributes from the plurality of network packets in the packet flow; and
including the extracted one or more event attributes in the first event stream.

8. The computer-implemented method of claim 1 , wherein generating the event stream from the packet flow further comprises:
identifying one or more event attributes associated with the first protocol from the configuration information;
extracting the one or more event attributes from the plurality of network packets in the packet flow;
transforming, based on the configuration information, the extracted one or more event attributes; and
including the transformed one or more event attributes in the first event stream.

9. The computer-implemented method of claim 1 , wherein the first protocol comprises at least one of: a first transport layer protocol, a first session layer protocol, a first presentation layer protocol, and a first application layer protocol, and wherein the second protocol comprises at least one of: a second transport layer protocol, a second session layer protocol, a second presentation lay protocol, and a second application layer protocol.

10. The computer-implemented method of claim 1 , wherein the first event stream is sent to a first component on the network, and wherein the second event stream is sent to a second component on the network that is different from the first component.

11. A remote capture agent, comprising:
a processor;
a non-transitory computer readable storage medium storing instructions which, when executed by the processor, cause the remote capture agent to:
monitor network traffic comprising a plurality of network packets on the network;
identify a packet flow in the plurality of network packets, wherein the packet flow is associated with a communication path between a source and a destination;
identify a first protocol and a second protocol associated with the packet flow;
generate, based on configuration information associated with the first protocol and the second protocol, a first event stream from the packet flow, wherein the first event stream comprises time-series event data created based on first data derived from network packets of the packet flow, and a second event stream from the packet flow, wherein the second event stream comprises time-series event data created based on second data derived from network packets of the packet flow; and
send the first event stream and the second event stream to another component on the network.

12. The remote capture agent of claim 11 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
obtain the configuration information from a configuration server over a network; and
use the configuration information to configure the generation of the first event stream and the second event stream during runtime of the remote capture agent.

13. The remote capture agent of claim 11 , wherein the packet flow is a first packet flow and the plurality of network packets is a first plurality of network packets, and wherein the instructions, when executed, further cause the remote capture agent to:
identify a second packet flow including a second plurality of network packets;
identify a third protocol associated with the second packet flow;
generate, based on configuration information associated with the third protocol, a third event stream from the second packet flow at the remote capture agent, wherein the third event stream comprises time-series event data created based on data derived from network packets of the second packet flow; and
transmit the third event stream to another component on the network.

14. The remote capture agent of claim 11 , wherein the instructions, when executed by the processor, further cause the remote capture agent to identify the plurality of network packets of the packet flow based on control information in the network packets.

15. The remote capture agent of claim 11 , wherein the instructions, when executed by the processor, further cause the remote capture agent to:
assemble the packet flow from the plurality of network packets; and
in response to detecting encryption of the network packets of the packet flow, decrypt the network packets in the packet flow prior to identifying the first protocol and the second protocol associated with the packet flow.

16. The remote capture agent of claim 11 , wherein the network packets of the packet flow are as

Assignees

• Splunk Inc

Inventors

• Shcherbakov Vladimir A
• Dickey Michael

Classifications

• H04L69/22 Primary

  Parsing or analysis of headers · CPC title

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title