Log event summarization for distributed server system

Patent metadata

Field	Value
Publication number	US-10268750-B2
Application number	US-201615011022-A
Country	US
Kind code	B2
Filing date	Jan 29, 2016
Priority date	Jan 29, 2016
Publication date	Apr 23, 2019
Grant date	Apr 23, 2019

Abstract

Clusters of log lines are identified based on log line templates. The log line templates are based on a punctuality pattern for a log line. Clusters of log lines that match each punctuality pattern can be identified based on comparisons between the log lines. The comparison may determine the similarity of the log lines and ultimately identify whether the log lines are close enough to be clustered. The comparison may be based on generated n-grams for the log lines and performing a hash on the n-grams. The resulting cluster information may be communicated to a user in an interface.

First claim

What is claimed is:

1. A method for monitoring log event data, comprising:
collecting log event data by one or more agents installed on one or more machines that perform a distributed business transaction;
determining, by the one or more agents, whether the log event data matches a punctuation pattern associated with one of a plurality of templates, wherein the punctuation pattern includes at least one delimiter and at least one field value;
grouping the log event data into a particular group based on which of the plurality of templates the log event data matches; and
generating clusters for the log event data within each group based on a distance calculation between log data within the particular group.

2. The method of claim 1 , further comprising:
routing the log event data to a particular remote machine based on which template of the plurality of templates the log event data matches, wherein the particular remote machine is associated with a particular template of the plurality of templates.

3. The method of claim 1 , wherein generating clusters includes determining a distance between each log event data within a particular group.

4. The method of claim 3 , wherein determining a distance includes generating an n-gram for each log event data.

5. The method of claim 3 , wherein determining a distance includes generating a hash for log event data.

6. The method of claim 1 , wherein generating clusters includes clustering two log event data together if the distance between the two log event data is within a threshold.

7. The method of claim 1 , further including reporting the clusters of log event data from multiple machines in a user interface.

8. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for monitoring log event data, the method comprising:
collecting log event data by one or more agents installed on one or more machines that perform a distributed business transaction;
determining, by the one or more agents, whether the log event data matches a punctuation pattern associated with one of a plurality of templates, wherein the punctuation pattern includes at least one delimiter and at least one field value;
grouping the log event data into a particular group based on which of the plurality of templates the log event data matches; and
generating clusters for the log event data within each group based on a distance calculation between log data within the particular group.

9. The non-transitory computer readable storage medium of claim 8 , the method further comprising:
routing the log event data to a particular remote machine based on which template of the plurality of templates the log event data matches, wherein the particular remote machine is associated with a particular template of the plurality of templates.

10. The non-transitory computer readable storage medium of claim 8 , wherein generating clusters includes determining a distance between each log event data within a particular group.

11. The non-transitory computer readable storage medium of claim 10 , wherein determining a distance includes generating an n-gram for each log event data.

12. The non-transitory computer readable storage medium of claim 10 , wherein determining a distance includes generating a hash for log event data.

13. The non-transitory computer readable storage medium of claim 8 , wherein generating clusters includes clustering two log event data together if the distance between the two log event data is within a threshold.

14. The non-transitory computer readable storage medium of claim 8 , further including reporting the clusters of log event data from multiple machines in a user interface.

15. A system for monitoring log event data, the system comprising:
a plurality of machines, each machine including a processor and memory,
one or more modules stored on each of the plurality machines, the one or more modules stored in memory and executable by a corresponding processor to:
receive log event data from a plurality of agents installed on the plurality of machines that perform a distributed business transaction,
determine whether the log event data matches a punctuation pattern associated with one of a plurality of templates, wherein the punctuation pattern includes at least one delimiter and at least one field value,
group the log event data into a particular group based on which of the plurality of templates the log event data matches, and
generate clusters for the log event data within each group based on a distance calculation between log data within the particular group.

16. The system of claim 15 , wherein one or more modules are further configured to:
route the log event data to a particular remote machine based on which template of the plurality of templates the log event data matches, wherein the particular remote machine is associated with a particular template of the plurality of templates.

17. The system of claim 15 , the one or more modules further executable to determine a distance between each log event data within a particular group.

18. The system of claim 17 , wherein determining a distance includes generating an n-gram for each log event data.

19. The system of claim 17 , wherein determining a distance includes generating a hash for log event data.

20. The system of claim 15 , the one or more modules further executable to reporting the clusters of log event data from multiple machines in a user interface.

Assignees

• Appdynamics Llc
• Cisco Tech Inc

Inventors

• Zhao Yuchen
• Iyer Arjun

Classifications

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title

• G06F16/35

  Clustering; Classification · CPC title

• G06F16/285 Primary

  Clustering or classification · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title