Streamlined transaction and dimension data collection

Patent metadata

Field	Value
Publication number	US-11048677-B1
Application number	US-201414503505-A
Country	US
Kind code	B1
Filing date	Oct 1, 2014
Priority date	Oct 1, 2014
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

Disclosed are methods, apparatuses and systems for collecting transactional and dimensional data. One implementation includes a configuration management database, a first data collection agent operating on a first monitored server to collect metadata associated with the first monitored server from the configuration management database using the first data collection agent, collect metric data from the first monitored server regarding operation of the first monitored server, wherein the metric data includes at least one metric collected from at least one application executing on the first monitored server; and assemble the collected metric data and at least part of the collected metadata into a packet for transmission from the first monitored server to a message bus; and a distributed database configured to receive the packet from the message bus and to store the collected metric data and metadata included in the packet.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more memories including instructions executable by the one or more processors to implement:
a configuration management database including a plurality of configuration items each having a plurality of attributes;
a first data collection agent operating on a first monitored server to:
collect metadata associated with the first monitored server from the configuration management database;
collect metric data from the first monitored server regarding operation of the first monitored server, including at least one metric from at least one application executing on the first monitored server; and
assemble the collected metric data and at least part of the collected metadata into a packet for transmission from the first monitored server to a message bus using a single pipeline wherein the packet and subsequent packets are sequentially transmitted and received via the single pipeline, wherein the single pipeline comprises a singular communications path such that the collected metric data, the at least part of the collected metadata, and other packets transmitted via the single pipeline are transmitted and received in sequential order;

an aggregation service that aggregates data of the packet and at least one other packet from the message bus based on at least one of a plurality of features associated with both the packet and the at least one other packet; and
a distributed database that stores the aggregated data based on the at least one of the plurality of features.

2. The system of claim 1 , wherein the metadata is first metadata, the metric data is first metric data, and the packet is a first packet, wherein the one or more memories further includes instructions executable by the one or more processors to implement:
a second data collection agent operating on a second monitored server to:
collect second metadata associated with the second monitored server from the configuration management database;
collect second metric data from the second monitored server regarding operation of the second monitored server, including at least one metric from at least one database application executing on the second monitored server; and
assemble the collected second metric data and at least part of the collected second metadata into a second packet for transmission from the second monitored server to the message bus using the single pipeline, wherein the first packet and the second packet are transmitted to the message bus sequentially via the single pipeline.

3. The system of claim 2 , wherein the at least one application executing on the first monitored server is a java virtual machine application node operating java software configured to use the at least one database application executing on the second monitored server.

4. The system of claim 2 , wherein at least some of the collected first and second metadata is dimensional data and at least some of the collected first and second metric data is fact data, wherein the single pipeline is provided between each of the first and the second monitored servers and the distributed database.

5. The system of claim 1 , wherein the instructions to collect metadata associated with the first monitored server from the configuration management database includes:
sending a subscription request from the first data collection agent to the configuration management database, wherein the subscription request includes an identification of the first monitored server;
periodically receiving, from the configuration management database, metadata associated with the first monitored server, wherein the configuration management database sends the metadata to the first data collection agent based on status updates associated with the first monitored server.

6. The system of claim 5 , wherein the metadata includes attributes that are associated with a configuration item associated with the first monitored server and that are stored in the configuration management database, and further includes attributes associated with at least one other configuration item having a relationship with the configuration item.

7. The system of claim 6 , wherein the at least one other configuration item is associated with an application server executing on the first monitored server and the relationship is a dependency between the application server and the first monitored server.

8. The system of claim 5 , wherein the subscription request further includes an identification of at least one application server having an associated configuration item in the configuration management database.

9. The system of claim 1 , wherein the collected metadata includes metadata associated with the at least one application executing on the first monitored server.

10. The system of claim 9 , wherein the collected metadata includes at least one of a software version, instance type, scheduler state, node status, or deployment model.

11. The system of claim 1 , wherein the collected metadata excludes metadata not associated with the first monitored server.

12. The system of claim 2 , wherein the first and the second data collection agents collect the first and the second metadata data using a respective listener that subscribes to changes in metadata attributes associated with the respective first and second monitored servers from the configuration management database.

13. The system of claim 12 , wherein the respective listener subscribes to changes by creating a business rule in the configuration management database for each of the metadata attributes, the business rule requesting a latest value of each of the metadata attributes when the changes are detected.

14. The system of claim 1 , wherein the one or more memories further includes instructions executable by the one or more processors to implement:
a predictive model configured to compare a subset of the collected metric data against a statistical measure of the collected metric data to detect a performance issue associated with the first monitored server when a count of a number of times that the subset of the collected metric data exceeds the statistical measure is greater than a predetermined threshold.

15. The system of claim 1 , wherein the plurality of features comprises at least one of a timestamp, a location, source information, or transaction information.

16. A method comprising:
collecting metadata associated with a first monitored server from a configuration management database, the configuration management database including a plurality of configuration items each having a plurality of attributes;
collecting metric data from the first monitored server regarding operation of the first monitored server, including at least one metric from at least one application executing on the first monitored server;
assembling the collected metric data and at least part of the collected metadata into a packet for transmission from the first monitored server to a message bus using a single pipeline, wherein the packet and subsequent packets are sequentially transmitted and received via the single pipeline, wherein the single pipeline comprises a singular communications path such that the collected metric data, the at least part of the collected metadata, and other packets transmitted via the single pipeline are transmitted and received in sequential order;
aggregating data of the packet and at least one other packet based on at least one of a plurality of features associated with both the packet and the at least one other packet;
storing the aggregated data based on the at least one of th

Assignees

• Servicenow Inc

Inventors

• Chandrashekar Sridhar
• Tucker Stephen Scott
• Vincent Kelly Patricia
• Mutzel Christopher Robert
• Polinati Chinna Babu
• Dang Kanwaldeep Kaur
• Balasubrahmanya Greeshma Kulamarva
• Ardeleanu Eugen
• Bushireddy Kiran Kumar

Classifications

• H04L43/04 Primary

  Processing captured monitoring data, e.g. for logfile generation · CPC title

• H04L41/046

  comprising network management agents or mobile agents therefor · CPC title

• G06F16/22 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title