Host-based flow aggregation

Patent metadata

Field	Value
Publication number	US-11398987-B2
Application number	US-201916520220-A
Country	US
Kind code	B2
Filing date	Jul 23, 2019
Priority date	Jul 23, 2019
Publication date	Jul 26, 2022
Grant date	Jul 26, 2022

Abstract

Some embodiments provide a novel method for collecting and reporting attributes of data flows associated with machines executing on a plurality of host computers to an analysis appliance. Each host computer, in some embodiments, is responsible for collecting and reporting attributes of data flows associated with machines executing on a host computer. In some embodiments, the host computer includes a flow exporter that processes and publishes flow data to the analysis appliance, a set of agents for collecting context data relating to the flows from machines executing on the host, a set of additional modules that provide additional context data, an anomaly detection engine that analyzes flow data and context data and provides additional context data, and a context exporter for processing and publishing context data to the analysis appliance.

First claim

We claim:

1. A method for collecting and reporting attributes of data flows associated with machines executing on a host computer, the method comprising:
at a host computer:
collecting statistics for individual flows associated with the machines executing on the host computer;
after each time period of a plurality of time periods:
identifying a plurality of groups of flows with each group comprising one or more of the individual flows;
for each identified group, identifying a set of attributes at least partly by aggregating at least a subset of the collected statistics of the individual flows in the group; and

after the plurality of time periods, providing the set of attributes for each group identified in the plurality of time periods to a server for further analysis of the data flows.

2. The method of claim 1 , wherein the identified set of attributes are contextual attributes for layers other than layers 2-4 of an open systems interconnection (OSI) model.

3. The method of claim 1 , wherein the identified set of attributes are contextual attributes for layers other than layers 2-7 of an OSI model.

4. The method of claim 1 , wherein the collected statistics are statistics generated on the host computer.

5. The method of claim 4 , wherein the collected statistics for a particular flow comprise at least one of a start time for the flow, a number of data messages exchanged, and a number of bytes exchanged.

6. The method of claim 1 , wherein the identified attributes comprise a version identifier for a current configuration of the machines executing on the host computer.

7. The method of claim 6 , wherein the version identifier is used to identify a set of service rules in effect at the time the statistics were collected for the flows of a particular identified group.

8. The method of claim 1 , wherein, for a particular group, identifying at least one attribute comprises concatenating values for the attribute from the one or more individual flows in each group.

9. The method of claim 8 , wherein the concatenated values comprise a plurality of unique values for the attribute from the one or more individual flows in each group.

10. The method of claim 1 , wherein, for a particular group, identifying at least one attribute comprises summing values from the one or more individual flows in each group.

11. The method of claim 1 , wherein, for a particular group, identifying at least one attribute comprises keeping an extreme value of the attribute from the one or more individual flows in each group.

12. The method of claim 11 , wherein the attribute comprises a start time and the extreme value is an earliest start time.

13. The method of claim 1 , wherein each individual flow is identified by a five-tuple comprising a source internet protocol (IP) address, a source port, a destination IP address, a destination port, and a transport layer protocol.

14. The method of claim 13 , wherein at least one group of the plurality of groups includes a plurality of individual flows that each have a same five-tuple and for which at least two individual flows have at least one attribute that is not identical.

15. The method of claim 14 , wherein the at least one attribute is a start time.

16. The method of claim 1 , wherein the statistics for at least one individual flow comprise a service rule used to process the at least one individual flow.

17. The method of claim 16 , wherein:
the individual flow is identified as part of a particular group; and
the set of attributes for the particular group comprises a value for a service rule identifier for the service rule and a configuration version identifier.

18. A non-transitory machine-readable medium storing a program which when executed by at least one processing unit of a host computer collects and reports attributes of data flows associated with machines executing on the host computer, the program comprising sets of instructions for:
collecting statistics for individual flows associated with the machines executing on the host computer;
after each time period of a plurality of time periods:
identifying a plurality of groups of flows with each group comprising one or more of the individual flows;
for each identified group, identifying a set of attributes at least partly by aggregating at least a subset of the collected statistics of the individual flows in the group; and

after the plurality of time periods, providing the set of attributes for each group identified in the plurality of time periods to a server for further analysis of the data flows.

19. The non-transitory machine-readable medium of claim 18 , wherein the identified set of attributes are contextual attributes for layers other than layers 2-4 of an open systems interconnection (OSI) model.

20. The non-transitory machine-readable medium of claim 18 , wherein the collected statistics for a particular flow comprise at least one of a start time for the flow, a number of data messages exchanged, and a number of bytes exchanged.

Assignees

• Vmware Inc

Inventors

• Jain Jayant
• Lu Russell
• Loi Ly
• Lund Rick
• Gopal Sushruth

Classifications

• H04L43/026 Primary

  using flow identification · CPC title

• H04L47/2441

  relying on flow classification, e.g. using integrated services [IntServ] · CPC title

• H04L47/2483

  involving identification of individual flows · CPC title

• H04L43/062

  related to network traffic · CPC title

• H04L43/04

  Processing captured monitoring data, e.g. for logfile generation · CPC title