Storing and querying metrics data

Patent metadata

Field	Value
Publication number	US-2018089286-A1
Application number	US-201615339863-A
Country	US
Kind code	A1
Filing date	Oct 31, 2016
Priority date	Sep 26, 2016
Publication date	Mar 29, 2018
Grant date	—

Abstract

The disclosed embodiments include a method performed by a data intake and query system to store and query metrics data. The method includes ingesting metrics, where each metric includes key values and numerical value indicative of a measured characteristic of a computing resource. The method further includes populating a first portion of a metric-series index (msidx) file with the key values and a second portion of the msidx file with numerical values indicative of a measured characteristic, where the first portion is distinct from the second portion. The method further includes receiving a query including criteria, evaluating the query by applying the criteria to the first portion of the msidx file to obtain query results indicative of metrics that satisfy the criteria, and displaying, on a display device, the query results or data indicative of the query results.

First claim

What is claimed is:

1 . A method, comprising:
receiving, by a data intake and query system, a query including criteria; evaluating, by the data intake and query system, the query by applying the criteria to a first portion of a metric-series index (msidx) file to obtain query results indicative of metric data that satisfy the criteria, wherein the first portion of the msidx file includes a plurality of key values of a metric and a second portion of the msidx file includes a numerical value of the metric, the numerical value being indicative of a measured characteristic of a computing resource, and the first portion being distinct from the second portion; and causing display, on a display device, of the query results or data indicative of the query results.

2 . The method of claim 1 , wherein the query is input by a user and expressed in a pipelined search language.

3 . The method of claim 1 , wherein the numerical value is a floating point value.

4 . The method of claim 1 , wherein the metric is semi-structured data or structured data.

5 . The method of claim 1 , wherein the characteristic of the computing resource is a utilization of a processor, a temperature of an electronic component, or a voltage reading of an electronic component.

6 . The method of claim 1 , wherein the metric is received by the data intake and query system over a computer network from a remote computer system.

7 . The method of claim 1 , wherein the query results are obtained by extracting data from a plurality of metrics stored in a journal distinct and separate from the msidx file, each location of each metric stored in the journal being referenced in the msidx file.

8 . The method of claim 1 , wherein the query results are obtained from the msidx file without retrieving data from a journal storing a plurality of metrics, the journal being separate and distinct from the msidx file.

9 . The method of claim 1 , wherein the metric has a plurality of dimensions including a plurality of required dimensions including respective key values and a plurality of optional dimensions that can each include a key value, and the criteria is indicative of a required dimension or an optional dimension.

10 . The method of claim 1 , wherein the metric has a plurality of required dimensions including respective key values, and the criteria is indicative of a required dimension.

11 . The method of claim 1 , wherein the metric has a time dimension including a value indicative of when the measured characteristic was measured, and a name dimension including a value indicative of a source of the measured characteristic, and the criteria is indicative of the time dimension or the name dimension.

12 . The method of claim 1 , wherein the metric has a plurality of optional dimensions, only some of the plurality of optional dimensions include values, and the criteria is indicative of an optional dimension.

13 . The method of claim 1 , wherein the metric has at least one of a host dimension, a manufacturer dimension, or a model dimension, and the criteria is indicative of the host dimension, the manufacturer dimension, or the model dimension.

14 . The method of claim 1 , wherein the metric has a dimension specified by a user prior to populating the msidx file with the plurality of key values and the numerical value, the criteria being indicative of the user specified dimension.

15 . The method of claim 1 , wherein the second portion of the msidx file includes a plurality of numerical values that are indicative of a time series of measured characteristics of a single computing resource.

16 . The method of claim 1 , further comprising:
ingesting, by the data intake and query system, raw machine data obtained over a computer network from a plurality of remote computer systems; generating, by the data intake and query system, a plurality of events indexed by timestamps, each of the plurality of events including a respective segment of the raw machine data; extracting, by the data intake and query system, a plurality of field values from the plurality of events, the plurality of field values satisfying the criteria; correlating, by the data intake and query system, the plurality of field values satisfying the criteria and the query results to obtain correlation results; and displaying, on the display device, the correlation results or data indicative of the correlation results.

17 . The method of claim 1 , wherein the query is a first query, the query results are first query results, and the criteria are first criteria, the method further comprising:
populating, by the data intake and query system, an acceleration table with the first query results and additional key values of metrics that satisfy the first criteria; receiving, by the data intake and query system, a second query including second criteria having a scope including the metrics that satisfy the first query; evaluating, by the data intake and query system, the second query by applying the second criteria to the acceleration table to obtain second query results without applying the second criteria to the msidx file; and displaying, on the display device, the second query results or data indicative of the second query results.

18 . The method of claim 1 , wherein the query is a first query, the query results are first query results, and the criteria are first criteria, the method further comprising:
populating, by the data intake and query system, an acceleration table with the first query results and all remaining key values of metrics that satisfy the first criteria as defined in at least one configuration file associated with metrics that satisfy the first criteria; receiving, by the data intake and query system, a second query including second criteria having a scope including the metrics that satisfy the first query; evaluating, by the data intake and query system, the second query by applying the second criteria to the acceleration table to obtain second query results without applying the second criteria to the msidx file; and displaying, on the display device, the second query results or data indicative of the second query results.

19 . The method of claim 1 , wherein the query is a first query input by a user as a first command and expressed in a pipelined search language, the query results are first query results, and the criteria are first criteria, the method further comprising:
populating, by the data intake and query system, an acceleration table with the first query results and additional key values of metrics that satisfy the first criteria; receiving, by the data intake and query system, a second query input by the user as a second command and expressed in a pipelined search language appended to the first command, the second query including second search criteria having a scope that includes the metrics that satisfy the first query; evaluating, by the data intake and query system, the second query by applying the second criteria to the acceleration table to obtain second query results without applying the second criteria to the msidx file; and displaying, on the display device, the second query results or data indicative of the second query results.

20 . The method of claim 1 , wherein the query is a first query, the query results are first query results, and the criteria are first criteria, the method further comprising:
populating, by the data intake and query system, an acceleration table with the first query results and additional key value

Assignees

• Splunk Inc

Inventors

• Marquardt David Ryan
• Yan Hailun
• Pride Christopher
• Patel Vishal

Classifications

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/9038

  Presentation of query results · CPC title

• G06F16/2228

  Indexing structures · CPC title