Network device implementing two-stage flow information aggregation

Patent metadata

Field	Value
Publication number	US-9906495-B2
Application number	US-201615356155-A
Country	US
Kind code	B2
Filing date	Nov 18, 2016
Priority date	Jul 30, 2014
Publication date	Feb 27, 2018
Grant date	Feb 27, 2018

Abstract

A network security device includes a network flow statistics processing engine to process network flow information related to network flows. The network flow statistics processing engine includes a first processing stage performing per-flow information aggregation and a second processing stage performing per-destination system component information aggregation, with each processing stage implementing a threshold-based data export scheme and a timer-based data export scheme. In this manner, up-to-date flow information is available to peer system components regardless of the varying flow rates of the network flow.

First claim

What is claimed is:

1. A network security device for processing a plurality of network flows, the network security device comprising:
a flow engine implemented in a hardware processor of the network security device and configured to receive incoming data packets associated with one or more network flows, the flow engine being configured to process and identify network flows associated with the received data packets and, based on the identified network flows, to forward the received data packets to respective packet processors for processing and to separately forward network flow information for statistics processing;
a network flow statistics processing engine implemented in the hardware processor of the network security device and configured to process the network flow information received from the flow engine and related to the network flows being handled by the flow engine, the network flow information comprising at least a flow identifier and count information of the received data packets for each network flow, the network flow statistics processing engine comprising:
a first processing stage configured to store and aggregate network flow information for each network flow handled by the flow engine on a per-flow basis, the first processing stage exporting the stored network flow information associated with a given network flow in response to a network flow information data for that network flow exceeding a flow information threshold or a first elapsed time for that network flow exceeding a flow timeout, the first elapsed time being a time duration from a first timestamp associated with that network flow and a current time; and
a second processing stage configured to receive the exported network flow information from the first processing stage, the second processing stage being configured to store the received network flow information on a per-destination basis into a per-destination storage, each destination being associated with a peer system component in the network security device and subscribing to the network flow information of one or more network flows, the second processing stage exporting the stored network flow information to a destination system component associated with a given destination in response to the destination having accumulated network flow information exceeding an accumulation threshold or a second elapsed time for that destination exceeding a destination timeout, the second elapsed time being a time duration from a second timestamp associated with that destination and a current time;
wherein the network flow statistics processing engine provides the network flow information to the subscribing destination system component of the network security device, the network flow information being used by the destination system component to perform management functions or to enforce security policy on the incoming data packets at the respective packet processors.

2. The network security device of claim 1 , wherein the first processing stage comprises a flow information table configured to store network flow information on a per-flow basis for each network flow being handled by the flow engine, the flow information table comprising a plurality of table entries, each table entry configured to store network flow information for one network flow.

3. The network security device of claim 1 , wherein the network flow information comprises a flow identifier, a packet count, and a byte count of the received data packets for each network flow.

4. The network security device of claim 1 , wherein the second processing stage comprises the per-destination storage configured to store network flow information exported by the first processing stage on the per-destination basis, each destination being configured to subscribe to one or more network flows and to receive network flow information for the one or more network flows to which the destination has subscribed.

5. The network security device of claim 4 , wherein the per-destination storage comprises a plurality of memory queues, each memory queue configured to store network flow information for one destination, each destination receiving network flow information associated with one or more subscribing network flows.

6. The network security device of claim 5 , wherein the per-destination storage comprises a first-in-first-out (FIFO) memory.

7. The network security device of claim 6 , wherein the per-destination storage is implemented as a first-in-first-out (FIFO) memory device external to and in communication with the network flow statistics processing engine.

8. The network security device of claim 1 , wherein the network flow information data comprises a packet count of the network flow and the flow information threshold comprises a packet count threshold, the first processing stage being configured to export the stored network flow information associated with a given network flow in response to the packet count for that network flow exceeding the packet count threshold.

9. The network security device of claim 1 , wherein the flow information threshold, the flow timeout, the accumulation threshold, and the destination timeout are programmable values.

10. The network security device of claim 1 , wherein the flow information threshold, the flow timeout, the accumulation threshold, and the destination timeout are selected based on the rate of the network flows being received by the flow engine.

11. The network security device of claim 1 , wherein the first processing stage is configured to assign a first flow information threshold to a first network flow and a second flow information threshold to a second network flow, the first flow information threshold different from the second flow information threshold.

12. The network security device of claim 1 , wherein the first processing stage is configured to assign a first flow timeout to a first network flow and a second flow timeout to a second network flow, the first flow timeout different from the second flow timeout.

13. The network security device of claim 1 , wherein the second processing stage is configured to assign a first accumulation threshold to a first network flow and a second accumulation threshold to a second network flow, the first accumulation threshold different from the second accumulation threshold.

14. The network security device of claim 1 , wherein the second processing stage is configured to assign a first destination timeout to a first network flow and a second destination timeout to a second network flow, the first destination timeout different from the second destination timeout.

15. The network security device of claim 1 , wherein the first timestamp has an initial time value being the time a network flow is added to the first processing stage and the first timestamp being updated in response to the first processing stage exporting the stored network flow information associated with a given network flow.

16. The network security device of claim 1 , wherein the second timestamp has an initial time value being the time the exported network flow information is received for a destination for the first time and the second timestamp is updated in response to the second processing stage exporting the stored network flow information to a destination system component.

17. A method for processing a plurality of network flows in a security device, the method comprising:
receiving incoming data packets at a flow engine of the security device, the security device including a hardware processor;
identifying, using the hardware processor, network flows associated with the recei

Assignees

• Palo Alto Networks Inc

Inventors

• Li Sidong
• Roberson William A
• Raghunath Savitha
• Ganesh Subramani
• Saharia Gyanesh

Classifications

• H04L63/0236

  Filtering by address, protocol, port number or service, e.g. IP-address or URL · CPC title

• H04L47/10 Primary

  Flow control; Congestion control · CPC title

• H04L63/02

  for separating internal from external traffic, e.g. firewalls · CPC title

• H04L2463/121

  Timestamp · CPC title

• H04L63/0263

  Rule management · CPC title