Packet capture ring: reliable, scalable packet capture for security applications

Patent metadata

Field	Value
Publication number	US-10044634-B2
Application number	US-201615225472-A
Country	US
Kind code	B2
Filing date	Aug 1, 2016
Priority date	Aug 1, 2016
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

Embodiments are directed to a packet capture ring that provides a single network tap for packet capture and a series of processors (or appliances) for handling serialization and search request processing in a confederated and highly scalable manner. One such appliance (a “primary” appliance) maintains a tap port to the network. Each packet capture appliance has a locally attached repository that stores raw packets and a juxtaposed index that allows for retrieval of those packets. The primary appliance sends a single copy of encapsulated packets in opposite directions around the ring to its descendants. A designation is made across the system as to a “currently designated” appliance for servicing requests for indexing and storage of captured packets. This current designation shifts from appliance to appliance in the system, as a “previously designated” appliance has its storage capacity filled.

First claim

We claim:

1. A computer implemented method comprising:
broadcasting, by a network tap appliance comprising a processor and configured to receive packets, a control protocol (CP) message to a plurality of appliances in a ring, the network tap appliance included in the plurality of appliances, wherein each appliance is associated with a repository and an index, and wherein each appliance includes a plurality of network interfaces;
obtaining, by the network tap appliance, a current capacity of a first repository in response to the broadcasted CP message;
altering, by the network tap appliance, a packet request from the first repository in response to the current capacity; and
in an event of failure to obtain, by the network tap appliance, a current capacity of a second repository, removing, by the network tap appliance, the second repository from an active packet forwarding designation list.

2. The method of claim 1 , wherein altering the packet request from the first repository is based on the current capacity of the first repository being equal to or approaching within a predetermined amount a capacity limit.

3. The method of claim 1 , further comprising:
designating, by the network tap appliance, a next repository as an active repository on the active packet forwarding designation list.

4. The method of claim 1 , further comprising:
re-inserting, by the network tap appliance, the second repository into the active forwarding designation list following receipt of a heartbeat from an associated appliance.

5. The method of claim 1 , further comprising:
sending, by the network tap appliance, a single copy of encapsulated packets in opposite directions around the ring to descendant appliances, the encapsulation comprising a timestamp as observed and marked by the network tap appliance and a forward header that designates an active appliance.

6. A system for packet capture and search, the system comprising:
a plurality of appliances arranged in a ring configuration, wherein each appliance is associated with a repository and an index, and wherein each appliance includes a plurality of network interfaces; and
a network tap appliance comprising a processor and configured to receive packets from a network via a network tap, the network tap appliance further configured to:
broadcast a control protocol (CP) message to the plurality of appliances in the ring;
obtain a current capacity of a first repository in response to the broadcasted CP message;
alter a packet request from the first repository in response to the current capacity; and
failing to obtain a current capacity of a second repository, remove the second repository from an active packet forwarding designation list.

7. The system of claim 6 , wherein altering the packet request from the first repository is based on the current capacity of the first repository being equal to or approaching within a predetermined amount a capacity limit.

8. The system of claim 6 , wherein the network tap appliance is further configured to:
designate a next repository as an active repository on the active packet forwarding designation list.

9. The system of claim 6 , wherein the network tap appliance is further configured to:
re-insert the second repository into the active forwarding designation list following receipt of a heartbeat from an associated appliance.

10. The system of claim 6 , wherein the network tap appliance is further configured to:
send a single copy of encapsulated packets in opposite directions around the ring to descendant appliances, the encapsulation comprising a timestamp as observed and marked by the network tap appliance and a forward header that designates an active appliance.

11. A computer implemented method for transmission of messages of a control protocol (CP) to established protocol peer appliances in a ring, wherein each appliance is configured to listen for and process network packet traffic transmitted on a primary network, such that the traffic is destined for any listening interface on the network; wherein each appliance in the ring is associated with a storage repository for captured packet traffic and is associated with an index; and wherein each appliance includes a plurality of network interfaces used in execution of the capture processing along with processing of the CP messages; the method comprising:
obtaining, by a network tap appliance comprising a processor and configured to initially receive the network packet traffic via a network tap, a current capacity of the repositories of the appliances of the ring;
altering, by the network tap appliance, a packet storage request from an initially designated repository in response to an initially designated current capacity; and
in cases of failing to obtain a second current capacity of a second repository, removing, by the network tap appliance, the second repository from an active packet forwarding designation list.

12. The method of claim 11 , further comprising:
re-inserting, by the network tap appliance, the second repository into the active forwarding designation list following receipt of a heartbeat from an associated appliance.

13. The method of claim 12 , further comprising:
performing, by the network tap appliance, packet processing on packets stored in the second repository upon re-insertion of the second repository into the active forwarding designation list.

14. The method of claim 11 , further comprising:
breaking, by the network tap appliance, a link between the network tap appliance and a last appliance of the ring;
linking, by the network tap appliance, a new appliance with the network tap appliance and the last appliance between respective ones of the plurality of network interfaces; and
inserting, by the network tap appliance, a respective forwarding designation of the new appliance into the active packet forwarding designation list.

15. The method of claim 11 , further comprising:
transmitting, by the network tap appliance, a CP search message to the appliances of the ring;
receiving, by the network tap appliance, relevant results from one or more of the appliances of the ring for a respective window of time retained by a respective one of the one or more of the appliances; and
merging, by the network tap appliance, the received relevant results by concatenating each of the relevant results.

16. A computer program product for transmission of messages of a control protocol (CP) to established protocol peer appliances in a ring, wherein each appliance is configured to listen for and process network packet traffic transmitted on a primary network, such that the traffic is destined for any listening interface on the network; wherein each appliance in the ring is associated with a storage repository for captured packet traffic and is associated with an index; and wherein each appliance includes a plurality of network interfaces used in execution of the capture processing along with processing of the CP messages; the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor that is part of a network tap appliance to cause the processor to:
obtain a current capacity of the repositories of the appliances of the ring;
alter a packet storage request from an initially designated repository in response to an initially designated current capacity; and
in cases of failing to obtain a second current capacity of a second repository, remove the second repository from an active packet forwarding designation list.

Assignees

• IBM
• Napatech As

Inventors

• Bird William A
• Couturier Russell
• Silliman Thomas D
• Tackabury Wayne
• Agerholm Alex Omo
• Lilja Michael Milde
• Ekner Peter Dahl
• Soeberg Philip Due

Classifications

• H04L47/34

  ensuring sequence integrity, e.g. using sequence numbers · CPC title

• H04L43/106

  using time related information in packets, e.g. by adding timestamps · CPC title

• H04L69/24

  Negotiation of communication capabilities · CPC title

• H04L2012/421

  Interconnected ring systems · CPC title

• H04L49/9084

  Reactions to storage capacity overflow · CPC title