Packet capture ring: reliable, scalable packet capture for security applications

Patent metadata

Field	Value
Publication number	US-10601729-B2
Application number	US-201816018973-A
Country	US
Kind code	B2
Filing date	Jun 26, 2018
Priority date	Aug 1, 2016
Publication date	Mar 24, 2020
Grant date	Mar 24, 2020

Abstract

Embodiments are directed to a packet capture ring that provides a single network tap for packet capture and a series of processors (or appliances) for handling serialization and search request processing in a confederated and highly scalable manner. One such appliance (a “primary” appliance) maintains a tap port to the network. Each packet capture appliance has a locally attached repository that stores raw packets and a juxtaposed index that allows for retrieval of those packets. The primary appliance sends a single copy of encapsulated packets in opposite directions around the ring to its descendants. A designation is made across the system as to a “currently designated” appliance for servicing requests for indexing and storage of captured packets. This current designation shifts from appliance to appliance in the system, as a “previously designated” appliance has its storage capacity filled.

First claim

We claim:

1. A computer implemented method comprising:
broadcasting, by a network tap appliance comprising a processor and configured to receive packets, a control protocol (CP) message to a plurality of appliances in a ring, the network tap appliance included in the plurality of appliances, wherein each appliance is associated with a repository and an index, and wherein each appliance includes a plurality of network interfaces;
obtaining, by the network tap appliance, a current capacity of a first repository in response to the broadcasted CP message; and
altering, by the network tap appliance, a packet request from the first repository in response to the current capacity being equal to or approaching within a predetermined amount capacity limit.

2. The method of claim 1 , further comprising:
designating, by the network tap appliance, a next repository as an active repository on the active packet forwarding designation list.

3. The method of claim 1 , further comprising:
sending, by the network tap appliance, an encapsulation of the packets in opposite directions around the ring to descendant appliances such that each of the descendant appliances receives two copies of the encapsulation.

4. The method of claim 3 , wherein the encapsulation comprises a timestamp as observed and marked by the network tap appliance and a forward header that designates an active appliance.

5. The method of claim 4 , further comprising:
ignoring, by the descendant appliances, a later-arriving of the two copies of the encapsulation.

6. A system for packet capture and search, the system comprising:
a plurality of appliances arranged in a ring configuration, wherein each appliance is associated with a repository and an index, and wherein each appliance includes a plurality of network interfaces; and
a network tap appliance comprising a processor and configured to receive packets from a network via a network tap, the network tap appliance further configured to:
broadcast a control protocol (CP) message to the plurality of appliances in the ring;
obtain a current capacity of a first repository in response to the broadcasted CP message;
alter a packet request from the first repository in response to the current capacity of the first repository being equal to or approaching within a predetermined amount capacity limit; and
designate a next repository as an active repository on the active packet forwarding designation list.

7. The system of claim 6 , wherein the network tap appliance is further configured to:
send an encapsulation of the packets in opposite directions around the ring to descendant appliances such that two copies of the encapsulation are received by each descendant appliance.

8. The system of claim 7 , wherein the encapsulation comprises a timestamp as observed and marked by the network tap appliance and a forward header that designates an active appliance.

9. The system of claim 6 , wherein the descendant appliances are further configured to:
ignore a later arriving encapsulation of the two copies.

10. A computer implemented method for transmission of messages of a control protocol (CP) to established protocol peer appliances in a ring, wherein each appliance is configured to listen for and process network packet traffic transmitted on a primary network, such that the traffic is destined for any listening interface on the network; wherein each appliance in the ring is associated with a storage repository for captured packet traffic and is associated with an index; and wherein each appliance includes a plurality of network interfaces used in execution of the capture processing along with processing of the CP messages, the method comprising:
obtaining, by a network tap appliance comprising a processor and configured to initially receive the network packet traffic via a network tap, a current capacity of the repositories of the appliances of the ring; and
altering, by the network tap appliance, a packet storage request from an initially designated repository in response to an initially designated current capacity based on the current capacity of the initially designated repository being equal to or approaching within a predetermined amount capacity limit.

11. The method of claim 10 , further comprising:
breaking, by the network tap appliance, a link between the network tap appliance and a last appliance of the ring;
linking, by the network tap appliance, a new appliance with the network tap appliance and the last appliance between respective ones of the plurality of network interfaces; and
inserting, by the network tap appliance, a respective forwarding designation of the new appliance into the active packet forwarding designation list.

12. The method of claim 10 , further comprising:
transmitting, by the network tap appliance, a CP search message to the appliances of the ring;
receiving, by the network tap appliance, relevant results from one or more of the appliances of the ring for a respective window of time retained by a respective one of the one or more of the appliances; and
merging, by the network tap appliance, the received relevant results by concatenating each of the relevant results.

13. The method of claim 10 , further comprising:
sending, by the network tap appliance, an encapsulation of the packets in opposite directions around the ring to descendant appliances such that each of the descendant appliances receives two copies of the encapsulation; and
performing, by each of the descendant appliances, a de-duplication process to remove a later-arriving of the two copies of the encapsulation.

14. A computer program product for transmission of messages of a control protocol (CP) to established protocol peer appliances in a ring, wherein each appliance is configured to listen for and process network packet traffic transmitted on a primary network, such that the traffic is destined for any listening interface on the network; wherein each appliance in the ring is associated with a storage repository for captured packet traffic and is associated with an index; and wherein each appliance includes a plurality of network interfaces used in execution of the capture processing along with processing of the CP messages; the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor that is part of a network tap appliance to cause the processor to:
obtain a current capacity of the repositories of the appliances of the ring; and
alter a packet storage request from an initially designated repository in response to an initially designated current capacity based on the current capacity of the initially designated repository being equal to or approaching within a predetermined amount capacity limit.

15. The computer program product of claim 14 , wherein the program instructions further cause the processor to:
break a link between the network tap appliance and a last appliance of the ring;
link a new appliance with the network tap appliance and the last appliance between respective ones of the plurality of network interfaces; and
insert a respective forwarding designation of the new appliance into the active packet forwarding designation list.

16. The computer program product of claim 14 , wherein the program instructions further cause the processor to:
transmit a CP search message to the appliances of the ring;
receive relevant results from one or more of the appliances of the ring for a respective window of time retained by a respective one of the one or more of the a

Assignees

• IBM
• Napatech As

Inventors

• Bird William A
• Couturier Russell
• Silliman Thomas D
• Tackabury Wayne
• Agerholm Alex Omo
• Lilja Michael Milde
• Ekner Peter Dahl
• Soeberg Philip Due

Classifications

• H04L47/78

  Architectures of resource allocation · CPC title

• H04L43/106

  using time related information in packets, e.g. by adding timestamps · CPC title

• H04L47/762 Primary

  triggered by the network · CPC title

• H04L43/0882

  Utilisation of link capacity · CPC title

• H04L2012/421

  Interconnected ring systems · CPC title