Unified error monitoring, alerting, and debugging of distributed systems

Patent metadata

Field	Value
Publication number	US-10810074-B2
Application number	US-201816226405-A
Country	US
Kind code	B2
Filing date	Dec 19, 2018
Priority date	Dec 19, 2018
Publication date	Oct 20, 2020
Grant date	Oct 20, 2020

Abstract

The disclosed embodiments provide a system for performing unified error monitoring, alerting, and debugging of distributed systems. During operation, the system receives logs of activity within a distributed system over event streams in a distributed streaming platform. Upon identifying an error in the received logs, the system collects, from the event streams, log entries generated within a time window of the error into a contextual log associated with the error. The system then outputs an alert containing metadata for accessing the contextual log.

First claim

What is claimed is:

1. A method, comprising:
receiving, by one or more computer systems, a plurality of logs of activity within a distributed system over a plurality of event streams in a distributed streaming platform;
upon identifying an error in the plurality of logs, collecting, by the one or more computer systems from the plurality of event streams, a plurality of log entries, generated within a time window of the error, into a contextual log associated with the error;
wherein the plurality of log entries includes (a) a first log entry from a first event stream of the plurality of event streams and (b) a second log entry from a second event stream of the plurality of event streams;
wherein collecting comprises:
searching the plurality of event streams for log entries that fall within the time window of the error and match an attribute associated with the error, wherein searching the plurality of event streams results in identifying the plurality of log entries; and
aggregating the plurality of log entries into the contextual log; and

outputting, by the one or more computer systems, an alert comprising metadata for accessing the contextual log.

2. The method of claim 1 , further comprising:
displaying a representation of the contextual log within a user interface; and
updating the displayed representation based on one or more filters specified through the user interface.

3. The method of claim 2 , wherein the representation of the contextual log comprises at least one of:
a visualization of events represented by the log entries;
a visual indication of a potential root cause of the error; or
a time-based ordering of the log entries.

4. The method of claim 2 , wherein the one or more filters comprise at least one of:
a log file name;
a host name;
a time;
a process;
a logging source;
a class of error; or
a duration of execution.

5. The method of claim 1 , wherein the log entries comprise:
an application log entry; or
a machine state entry.

6. The method of claim 5 , wherein the machine state entry comprises at least one of:
processes running on a machine;
a resource usage;
a network status; or
a socket status.

7. The method of claim 1 , wherein the error comprises at least one of:
an exception;
a logged error; or
a core dump.

8. The method of claim 1 , wherein the metadata comprises at least one of:
a link to the contextual log; or
a path to a location of the contextual log.

9. A system, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
receive a plurality of logs of activity within a distributed system over a plurality of event streams in a distributed streaming platform;
upon identifying an error in the plurality of logs, collect, from the plurality of event streams, into a contextual log associated with the error, a plurality of log entries generated within a time window of the error;
wherein the plurality of log entries includes (a) a first log entry from a first event stream of the plurality of event streams and (b) a second log entry from a second event stream of the plurality of event streams;
wherein collecting comprises:
searching the plurality of event streams for log entries that fall within the time window of the error and match an attribute associated with the error, wherein searching the plurality of event streams results in identifying the plurality of log entries; and
aggregating the plurality of log entries into the contextual log; and

output an alert comprising metadata for accessing the contextual log.

10. The system of claim 9 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the system to:
display a representation of the contextual log within a user interface; and
update the displayed representation based on one or more filters specified through the user interface.

11. The system of claim 10 , wherein the representation of the contextual log comprises at least one of:
a visualization of events represented by the log entries;
a visual indication of a potential root cause of the error; or
a time-based ordering of the log entries.

12. The system of claim 10 , wherein the one or more filters comprise at least one of:
a log file name;
a host name;
a time;
a process;
a logging source;
a class of error; or
a duration of execution.

13. The system of claim 9 , wherein collecting the log entries comprises:
standardizing metadata fields in the log entries.

14. The system of claim 13 , wherein the metadata fields comprise at least one of:
a timestamp;
a host name;
a cluster name; or
a log file name.

15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
receiving a plurality of logs of activity within a distributed system over a plurality of event streams in a distributed streaming platform;
upon identifying an error in the the plurality of logs, collecting, from the plurality of event streams, a plurality of log entries generated within a time window of the error into a contextual log associated with the error;
outputting an alert comprising metadata for accessing the contextual log;
displaying a representation of the contextual log within a user interface; and
updating the displayed representation based on one or more filters specified through the user interface.

16. The method of claim 15 , wherein collecting the log entries within the time window of the error into the contextual log associated with the error comprises:
searching the event streams for the log entries that fall within the time window of the error and match an attribute associated with the error; and
aggregating the log entries into the contextual log.

17. The method of claim 16 , wherein collecting the log entries within the time window of the error into the contextual log associated with the error further comprises:
standardizing metadata fields in the log entries prior to aggregating the log entries into the contextual log.

18. The method of claim 17 , wherein the metadata fields comprise at least one of:
a timestamp;
a host name;
a cluster name; or
a log file name.

19. The method of claim 16 , wherein the attribute comprises at least one of:
a cluster in which the error was generated;
a tree of service calls related to the error; or
a session associated with the error.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the representation of the contextual log comprises at least one of:
a visualization of events represented by the log entries;
a visual indication of a potential root cause of the error; or
a time-based ordering of the log entries.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yu Yan
• Constandache Ionut
• Miller Brent D
• Meyer Scott M
• Arsintescu Bogdan G

Classifications

• G06F11/32

  with visual {or acoustical} indication of the functioning of the machine · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F11/3086

  where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• G06F11/3082

  the data filtering being achieved by aggregating or compressing the monitored data · CPC title