Generating and Storing Summarization Tables for Sets of Searchable Events

Patent metadata

Field	Value
Publication number	US-2016004750-A1
Application number	US-201514815973-A
Country	US
Kind code	A1
Filing date	Aug 1, 2015
Priority date	Jan 31, 2013
Publication date	Jan 7, 2016
Grant date	—

Abstract

Embodiments are directed are towards the transparent summarization of events. Queries directed towards summarizing and reporting on event records may be received at a search head. Search heads may be associated with one more indexers containing event records. The search head may forward the query to the indexers the can resolve the query for concurrent execution. If a query is a collection query, indexers may generate summarization information based on event records located on the indexers. Event record fields included in the summarization information may be determined based on terms included in the collection query. If a query is a stats query, each indexer may generate a partial result set from previously generated summarization information, returning the partial result sets to the search head. Collection queries may be saved and scheduled to run and periodically update the summarization information.

First claim

What is claimed as new and desired to be protected by Letters Patent of the United States is:

1 . A method, comprising:
creating two or more sets of searchable, time stamped event records from raw data stored in at least one data store, wherein each searchable, time stamped event record in the two or more sets of searchable, time stamped event records includes a portion of the raw data and is associated with a time stamp derived from the raw data, wherein the raw data reflects activity in an information technology environment; generating a summarization table for each set of searchable, time stamped event records in the two or more sets of searchable, time stamped event records that:
identifies one or more field values, wherein a field value comprises a value that appears in an associated field in one or more searchable, time stamped event records in the set of searchable, time stamped event records; and
for each field value, identifies the one or more searchable, time stamped event records in the set of searchable, time stamped event records that contain the field value for the associated field;
storing the summarization table for each set of searchable, time stamped event records among the two or more sets of time stamped searchable event records; selecting a stored summarization table based on a received query that includes search criteria for evaluating field values for one or more fields; using the search criteria to evaluate field values for one or more fields in the selected summarization table to generate a query result; and wherein the query result reflects an aspect of activity in the information technology environment.

2 . The method of claim 1 , further comprising:
causing display of information based on the query result.

3 . The method of claim 1 , further comprising:
causing display of the query result.

4 . The method of claim 1 , further comprising:
executing an action based on the query result.

5 . The method of claim 1 , further comprising:
based on using the search criteria to evaluate field values for one or more fields in the selected summarization table, retrieving event records identified in the summarization table for further processing to generate the query result.

6 . The method of claim 1 , wherein the query result is generated from the summarization table with retrieving event records identified in the summarization table.

7 . The method of claim 1 , further comprising:
storing the two or more sets of searchable, time stamped event records in an indexed data store.

8 . The method of claim 1 , further comprising:
storing the two or more sets of searchable, time stamped event records in an indexed data store; wherein the indexed data store comprises a distributed indexed data store.

9 . The method of claim 1 , further comprising:
storing the two or more sets of searchable, time stamped event records in an indexed data store; wherein the indexed data store is stored in a distributed manner among two or more servers.

10 . The method of claim 1 , further comprising:
receiving a command identifying fields to include in the summarization table for each set of searchable, time stamped event records in the two or more sets of searchable, time stamped event records.

11 . The method of claim 1 , wherein the raw data includes machine data.

12 . The method of claim 1 , wherein the raw data includes log data.

13 . The method of claim 1 , wherein the raw data includes unstructured data.

14 . The method of claim 1 , wherein the summarization table for each set of searchable, time stamped event records in the one or more sets of searchable, time stamped event records includes two or more table portions, and wherein the two or more table portions are stored in a distributed manner.

15 . The method of claim 1 , wherein the summarization table for each set of searchable, time stamped event records in the one or more sets of searchable, time stamped event records includes two or more table portions, and wherein each of the two or more table portions is stored in a distributed manner proximate to a subset of the event records to which it pertains.

16 . An apparatus, comprising:
an event record creator, implemented at least partially in hardware, that creates two or more sets of searchable, time stamped event records from raw data stored in at least one data store, wherein each searchable, time stamped event record in the two or more sets of searchable, time stamped event records includes a portion of the raw data and is associated with a time stamp derived from the raw data, wherein the raw data reflects activity in an information technology environment; a summarization table generator, implemented at least partially in hardware, that generates a summarization table for each set of searchable, time stamped event records in the two or more sets of searchable, time stamped event records that:
identifies one or more field values, wherein a field value comprises a value that appears in an associated field in one or more searchable, time stamped event records in the set of searchable, time stamped event records; and
for each field value, identifies the one or more searchable, time stamped event records in the set of searchable, time stamped event records that contain the field value for the associated field;
a summarization table storage system, implemented at least partially in hardware, that stores the summarization table for each set of searchable, time stamped event records among the two or more sets of time stamped searchable event records; a summarization table selector, implemented at least partially in hardware, that selects a stored summarization table based on a received query that includes search criteria for evaluating field values for one or more fields; a subsystem, implemented at least partially in hardware, that uses the search criteria to evaluate field values for one or more fields in the selected summarization table to generate a query result; and wherein the query result reflects an aspect of activity in the information technology environment.

17 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardware, that causes display of information based on the query result.

18 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardware, that causes display of the query result.

19 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardware, that executes an action based on the query result.

20 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardware, that based on using the search criteria to evaluate field values for one or more fields in the selected summarization table, retrieves event records identified in the summarization table for further processing to generate the query result.

21 . The apparatus of claim 16 , wherein the query result is generated from the summarization table with retrieving event records identified in the summarization table.

22 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardware, that stores the two or more sets of searchable, time stamped event records in an indexed data store.

23 . The apparatus of claim 16 , further comprising:
a subsystem, implemented at least partially in hardwar

Assignees

• Splunk Inc

Inventors

• Marquardt David Ryan
• Sorkin Stephen Phillip
• Zhang Steve Yu

Classifications

• G06F16/2228 Primary

  Indexing structures · CPC title

• G06F16/24539

  using cached or materialised query results · CPC title

• G06F16/00

  Information retrieval; Database structures therefor; File system structures therefor · CPC title

• G06F16/951

  Indexing; Web crawling techniques · CPC title

• G06F16/284

  Relational databases · CPC title