Distributed network data management systems and methods

Patent metadata

Field	Value
Publication number	US-12335126-B2
Application number	US-202418615347-A
Country	US
Kind code	B2
Filing date	Mar 25, 2024
Priority date	Sep 7, 2021
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

Systems and methods for application performance management across one or more networks are disclosed. A system includes a plurality of geographically distributed computing devices executing one or more applications. A plurality of collectors are distributed across the one or more networks, each collector being positioned proximate a respective computing device. The collectors may sample, by each of the plurality of collectors, a plurality of trace events received from the applications executing on the computing devices proximate the collector. The collectors may retain in memory sampled trace events that meet a configurable attribute. The collectors may use probabilistic sets and tail-based sampling to determine root events and policies for identifying relevant traces. Benefits can be achieved in bandwidth savings, network cost and cyber security.

First claim

What is claimed is:

1. A method of application performance monitoring in one or more networks, comprising:
executing a plurality of respective applications on two or more computing devices located in at least two different geographical regions connected via the one or more networks;
distributing a plurality of collectors across the one or more networks, each collector being positioned proximate a respective computing device of the two or more computing devices;
sampling, by each of the plurality of collectors, a plurality of trace events received from the applications on the computing devices proximate the respective collector;
retaining in local memory sampled trace events that meet a configurable attribute comprising at least one of a latency threshold for completing a trace associated with one or more of the sampled trace events, or a success or failure of the trace,
selectively transmitting by one of the collectors across the one or more networks a representation of the retained trace events to others of the plurality of collectors;
transmitting to an orchestration server across the one or more networks information comprising a trace event identified based on a local sampling decision, the orchestration server configured for indexing other information representing sampled trace events obtained from other collectors of the plurality of collectors that correspond with the trace event identified in the information; and
publishing sampled trace identifiers (IDs) to the orchestration server to allow other collectors to enable indexing of events related to the trace.

2. The method of claim 1 , wherein:
selectively transmitting the representation of the retained sampled trace events further comprises transmitting the representation via an orchestration server; and
the orchestration server transmits the representation to the others using the one or more networks.

3. The method of claim 2 , wherein the representation comprises a trace identifier (ID).

4. The method of claim 1 , further comprising sampling by at least one of the collectors a root trace event.

5. The method of claim 1 , further comprising determining by the at least one of the collectors that the sampled trace event comprises a root trace event.

6. The method of claim 1 , wherein the sampling by at least one of the collectors comprises tail-based sampling.

7. The method of claim 6 , further comprising:
determining, using the tail-based sampling, at least one of a root transaction duration or a root transaction outcome; and
retaining a sampled trace event based at least in part on the determining.

8. The method of claim 1 , further comprising:
reservoir-sampling, by at least one of the collectors, a root transaction; and
synchronizing prior sampling decisions by the at least one of the collectors to index any other trace events that are part of a sampled trace.

9. A system for distributed data management, comprising:
a plurality of computing devices geographically distributed and communicatively coupled together via one or more networks, each computing device being configured to execute at least one application;
a plurality of geographically-distributed collectors, each collector being located proximate one or more of the computing devices and configured to:
sample a plurality of trace events from the one or more of the computing devices; and
retain in local memory sampled trace events that meet a configurable attribute comprising at least one of a latency threshold for completing a trace associated with one or more of the sampled trace events, or a success or failure of the trace,

wherein at least some collectors of the plurality of collectors are configured to:
determine a root event of a trace,
pass the root event through a reservoir sampler for making a local sampling decision,
transmit to an orchestration server across the one or more networks information comprising a trace event identified based on the local sampling decision, the orchestration server being configured to index other information representing sampled trace events obtained from other collectors of the plurality of collectors that correspond with the trace event identified in the information, and
publish sampled trace identifiers (IDs) to the orchestration server to allow other collectors to enable indexing of events related to the trace.

10. The system of claim 9 , further comprising an orchestration server, wherein at least some collectors of the plurality of collectors are further configured to:
generate a probabilistic set of trace identifiers (IDs); and
transmit the probabilistic set to the orchestration server.

11. The system of claim 10 , wherein the probabilistic set comprises a Bloom Filter.

12. The system of claim 9 , wherein at least some collectors of the plurality of collectors are further configured to:
perform tail-based sampling of event data to determine a property of a completed trace; and
locally store the trace or portion thereof when the property meets a threshold.

13. The system of claim 12 , wherein the property comprises at least one of a root transaction duration or a trace success or failure outcome.

14. The system of claim 12 , wherein the at least some collectors are further configured to synchronize sampling decisions to index any other trace events that are part of sampled traces.

15. The system of claim 9 , further comprising determining, by at least some collectors of the plurality of collectors, a root event of a plurality of trace events.

16. The system of claim 9 , wherein the at least some collectors of the plurality of collectors are configured to identify the root event by transforming sampled trace events into a directed acyclic graph (DAG) or a hierarchical tree.

Assignees

• Elasticsearch Bv

Inventors

• Wilkins Andrew
• Cohen Ron

Classifications

• H04L41/0686

  Additional information in the notification, e.g. enhancement of specific meta-data · CPC title

• H04L43/10 Primary

  Active monitoring, e.g. heartbeat, ping or trace-route · CPC title

• H04L41/069 Primary

  using logs of notifications; Post-processing of notifications · CPC title