Storing and querying metrics data

Patent metadata

Field	Value
Publication number	US-10642852-B2
Application number	US-201615339863-A
Country	US
Kind code	B2
Filing date	Oct 31, 2016
Priority date	Sep 26, 2016
Publication date	May 5, 2020
Grant date	May 5, 2020

Abstract

The disclosed embodiments include a method performed by a data intake and query system to store and query metrics data. The method includes ingesting metrics, where each metric includes key values and numerical value indicative of a measured characteristic of a computing resource. The method further includes populating a first portion of a metric-series index (msidx) file with the key values and a second portion of the msidx file with numerical values indicative of a measured characteristic, where the first portion is distinct from the second portion. The method further includes receiving a query including criteria, evaluating the query by applying the criteria to the first portion of the msidx file to obtain query results indicative of metrics that satisfy the criteria, and displaying, on a display device, the query results or data indicative of the query results.

First claim

What is claimed is:

1. A data intake and query system, comprising:
a processor; and
memory containing instructions that, when executed by the data intake and query system, cause the data intake and query system to:
ingest a plurality of metrics, each metric including a plurality of key values and numerical value indicative of a measured characteristic of a computing resource;
populate a first portion of a metric-series index (msidx) file with the plurality of key values and a second portion of the msidx file with every numerical value indicative of a measured characteristic, the first portion being distinct from the second portion;
receive a query including search criteria and statistics criteria;
evaluate the query by applying the search criteria to the first portion of the msidx file to obtain search results indicative of metrics that satisfy the search criteria;
populate an acceleration table with the search results and additional key values of the metrics that satisfy the search criteria;
evaluate the query by applying the statistics criteria to the acceleration table to obtain statistics results without applying the statistics criteria to the msidx file; and
cause display, on a display device, of the statistics results or data indicative of the statistics results.

2. The system of claim 1 , wherein the query is input by a user and expressed in a pipelined search language.

3. The system of claim 1 , wherein each numerical value is a floating point value.

4. The system of claim 1 , wherein each of the plurality of metrics is semi-structured data or structured data.

5. The system of claim 1 , wherein each characteristic of the computing resource is a utilization of a processor, a temperature of an electronic component, or a voltage reading of an electronic component.

6. The system of claim 1 , wherein the plurality of metrics are obtained by the data intake and query system over a computer network from a plurality of remote computer systems.

7. The system of claim 1 , wherein the search results and statistical results are obtained without extracting data from a journal storing the plurality of metrics, the journal being separate and distinct from the msidx file.

8. The system of claim 1 , wherein the acceleration table includes all key values defined in at least one configuration file associated with the metrics that satisfy the search criteria.

9. The system of claim 1 , wherein each metric has a plurality of dimensions including a plurality of required dimensions including respective key values and a plurality of optional dimensions that can each include a key value, wherein only some of the plurality of metrics include values for optional dimensions, and the search criteria is indicative of a required dimension or an optional dimension.

10. The system of claim 1 , wherein the metric has a plurality of required dimensions including respective key values, and the search criteria is indicative of a required dimension.

11. A method comprising:
ingesting, by a data intake and query system, a plurality of metrics, each metric including a plurality of key values and numerical value indicative of a measured characteristic of a computing resource;
populating, by the data intake and query system, a first portion of a metric-series index (msidx) file with the plurality of key values and a second portion of the msidx file with every numerical value indicative of a measured characteristic, the first portion being distinct from the second portion;
receiving, by the data intake and query system, a query including search criteria and statistics criteria;
evaluating, by the data intake and query system, the query by applying the search criteria to the first portion of the msidx file to obtain search results indicative of metrics that satisfy the search criteria;
populating, by the data intake and query system, an acceleration table with the search results and additional key values of the metrics that satisfy the search criteria;
evaluating, by the data intake and query system, the query by applying the statistics criteria to the acceleration table to obtain statistics results without applying the statistics criteria to the msidx file; and
causing display, on a display device, of the statistics results or data indicative of the statistics results.

12. The method of claim 11 , wherein the query is input by a user and expressed in a pipelined search language.

13. The method of claim 11 , wherein each numerical value is a floating point value.

14. The method of claim 11 , wherein each of the plurality of metrics is semi-structured data or structured data.

15. The method of claim 11 , wherein each characteristic of the computing resource is a utilization of a processor, a temperature of an electronic component, or a voltage reading of an electronic component.

16. The method of claim 11 , wherein the plurality of metrics are obtained by the data intake and query system over a computer network from a plurality of remote computer systems.

17. The method of claim 11 , wherein the search results and statistical results are obtained without extracting data from a journal storing the plurality of metrics, the journal being separate and distinct from the msidx file.

18. The method of claim 11 , wherein the acceleration table includes all key values defined in at least one configuration file associated with the metrics that satisfy the search criteria.

19. The method of claim 11 , wherein each metric has a plurality of dimensions including a plurality of required dimensions including respective key values and a plurality of optional dimensions that can each include a key value, wherein only some of the plurality of metrics include values for optional dimensions, and the search criteria is indicative of a required dimension or an optional dimension.

20. The method of claim 11 , wherein the metric has a plurality of required dimensions including respective key values, and the search criteria is indicative of a required dimension.

21. A non-transitory machine-readable storage medium storing instructions, an execution of which in a data intake and query system causes the data intake and query system to perform operations comprising:
ingesting a plurality of metrics, each metric including a plurality of key values and numerical value indicative of a measured characteristic of a computing resource;
populating a first portion of a metric-series index (msidx) file with the plurality of key values and a second portion of the msidx file with every numerical value indicative of a measured characteristic, the first portion being distinct from the second portion;
receiving a query including search criteria and statistics criteria;
evaluating the query by applying the search criteria to the first portion of the msidx file to obtain search results indicative of metrics that satisfy the search criteria;
populating an acceleration table with the search results and additional key values of the metrics that satisfy the search criteria;
evaluating the query by applying the statistics criteria to the acceleration table to obtain statistics results without applying the statistics criteria to the msidx file; and
causing display, on a display device, of the statistics results or data indicative of the statistics results.

22. The non-transitory machine-readable storage medium of claim 21 , wherein the query is input by a user and expressed in a pipelined search language.

23. The non-tr

Assignees

• Splunk Inc

Inventors

• Marquardt David Ryan
• Yan Hailun
• Pride Christopher
• Patel Vishal

Classifications

• G06F16/9535

  Search customisation based on user profiles and personalisation · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/2228

  Indexing structures · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title