Linking related events for various devices and services in computer log files on a centralized server

Patent metadata

Field	Value
Publication number	US-12147295-B2
Application number	US-202217817170-A
Country	US
Kind code	B2
Filing date	Aug 3, 2022
Priority date	Dec 15, 2017
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

A system with an interactive user interface for users to view and interact with sanitized log data received from a plurality of hosts, such as those associated with various services of an organization. The system may receive from hosts log files and/or metadata that have been filtered by agents executing on the respective hosts to remove or anonymize any sensitive or confidential information prior to transmission to the system. In some embodiments the system does further filtering of the sanitized data. Received sanitized data is parsed, indexed, and/or otherwise processed for optimal searching, and stored in a log pipeline. The system causes display of an electronic visualization interface.

First claim

What is claimed is:

1. A computing system comprising:
one or more non-transitory computer readable storage mediums configured to store a plurality of computer executable instructions; and
one or more hardware computer processors in communication with the one or more non-transitory computer readable storage mediums and configured to execute the plurality of computer executable instructions to cause the computing system to:
electronically receive and store, via one or more networks from a plurality of remote hosts, log files, wherein the plurality of remote hosts are configured to prepare log files in a uniform format that is in compliance with one or more configurable rules, wherein each of a subset of the log files includes one or more identifiers associated with a subset of a plurality of services;
based at least in part on the subset of the log files, determine lengths of processing times corresponding to one or more processes of a first service of the subset of the plurality of services, wherein the lengths of processing times indicate lengths of time it takes to execute respective processes of the one or more processes by the plurality of remote hosts;
cause display of one or more electronic visualization interfaces comprising:
a first graph comprising (1) nodes representing each service of the subset of the plurality of services that are associated with the one or more identifiers, and (2) edges connecting the nodes and representing flows of traffic between services of the subset of the plurality of services, wherein the first graph includes at least a first node representing the first service of the subset of the plurality of services; and
a second graph comprising a subset of the one or more processes of the first service of the subset of the plurality of services and corresponding lengths of processing times;

receive, via the second graph, user input indicating interaction with a first process of the subset of the one or more processes; and
in response to receiving the user input, update the second graph to display
one or more sub-processes within the first process, and corresponding lengths of processing times of the one or more sub-processes within the first process.

2. The computing system of claim 1 , wherein the first graph comprises a flame chart.

3. The computing system of claim 1 , wherein the lengths of processing times are measured in milliseconds.

4. The computing system of claim 1 , wherein the first graph lists the subset of the one or more processes in a ranked order based at least in part on the corresponding lengths of processing times.

5. The computing system of claim 1 , wherein the first graph lists a portion of the subset of the one or more processes that correspond to higher processing times.

6. The computing system of claim 1 , wherein the plurality of computer executable instructions are configured to further cause the computing system to:
determine calls performed by the one or more processes.

7. The computing system of claim 6 , wherein the second graph comprises:
a second node representing the first process of the subset of the one or more processes;
a third node representing a second process of the subset of the one or more processes; and
an edge connecting the second node and the third node, the edge indicating at least a number of calls being made by the first process to the second process.

8. The computing system of claim 6 , wherein the calls include one or more of: a quantity of calls being made by processes to other processes; specific calls that are being made for a specific service; and an indicator of a length of time to execute one or more calls.

9. The computing system of claim 1 , wherein the first graph comprises color indicators based at least in part on the corresponding lengths of processing times.

10. The computing system of claim 1 , wherein the one or more electronic visualization interfaces further comprises a log freshness indicator indicating a time or range of times that one or more logs were generated or received by the computing system.

11. The computing system of claim 10 , wherein the one or more logs include one or more of: event logs; trace logs; and service logs.

12. A computer-implemented method comprising, by one or more hardware processors executing program instructions:
electronically receiving and storing, via one or more networks from a plurality of remote hosts, log files, wherein the plurality of remote hosts are configured to prepare log files in a uniform format that is in compliance with one or more configurable rules, wherein each of a subset of the log files includes one or more identifiers associated with a subset of a plurality of services;
based at least in part on the subset of the log files, determining lengths of processing times corresponding to one or more processes of a first service of the subset of the plurality of services, wherein the lengths of processing times indicate lengths of time it takes to execute respective processes of the one or more processes by the plurality of remote hosts;
causing display of one or more electronic visualization interfaces comprising:
a first graph comprising (1) nodes representing each service of the subset of the plurality of services that are associated with the one or more identifiers, and (2) edges connecting the nodes and representing flows of traffic between services of the subset of the plurality of services, wherein the first graph includes at least a first node representing the first service of the subset of the plurality of services; and
a second graph comprising a subset of the one or more processes of the first service of the subset of the plurality of services and corresponding lengths of processing times;

receiving, via the second graph, user input indicating interaction with a first process of the subset of the one or more processes; and
in response to receiving the user input, updating the second graph to display
one or more sub-processes within the first process, and corresponding lengths of processing times of the one or more sub-processes within the first process.

13. The computer-implemented method of claim 12 , wherein the first graph comprises a flame chart.

14. The computer-implemented method of claim 12 , wherein the lengths of processing times are measured in milliseconds.

15. The computer-implemented method of claim 12 , wherein the first graph lists the subset of the one or more processes in a ranked order based at least in part on the corresponding lengths of processing times.

16. The computer-implemented method of claim 12 , further comprising:
determining calls performed by the one or more processes.

17. The computer-implemented method of claim 16 , wherein the second graph further comprises:
a second node representing the first process of the subset of the one or more processes;
a third node representing a second process of the subset of the one or more processes; and
an edge connecting the second node and the third node, the edge indicating at least a number of calls being made by the first process to the second process.

18. The computer-implemented method of claim 12 , wherein the first graph comprises color indicators based at least in part on the corresponding lengths of processing times.

19. The computer-implemented method of claim 12 , wherein the one or more electronic visualization interfaces further comprises a log freshness indicator indicating a time or range of times that one or more logs were generated or received by a computing system.

Assignees

• Palantir Technologies Inc

Inventors

• Abdelsalam Mahmoud
• Santos Eric
• Dearment Greg
• Wu Grant
• Ross James
• Simons Kevin
• Xu Guodong
• Wang Peter
• Niemi Richard
• Lakshmanan Vivek

Classifications

• G06F21/6254 Primary

  by anonymising data, e.g. decorrelating personal data from the owner's identification · CPC title

• G06F11/0787

  Storage of error reports, e.g. persistent data storage, storage using memory protection · CPC title

• G06F11/3072

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title

• G06F11/3068

  where the reporting involves data format conversion · CPC title

• G06F16/901

  Indexing; Data structures therefor; Storage structures (for retrieval from the web G06F16/951) · CPC title