Normalization of Time Stamps for Event Data

Patent metadata

Field	Value
Publication number	US-2016140238-A1
Application number	US-201615007176-A
Country	US
Kind code	A1
Filing date	Jan 26, 2016
Priority date	Oct 5, 2006
Publication date	May 19, 2016
Grant date	—

Abstract

Methods and apparatus consistent with the invention provide the ability to organize, index, search, and present time series data based on searches. Time series data are sequences of time stamped records occurring in one or more usually continuous streams, representing some type of activity. In one embodiment, time series data is organized into discrete events with normalized time stamps and the events are indexed by time and keyword. A search is received and relevant event information is retrieved based in whole or in part on the time indexing mechanism, keyword indexing mechanism, or statistical indices calculated at the time of the search.

First claim

1 . A method, comprising:
detecting whether time information is present in an event that includes a portion of raw data; in response to detecting that the time information is present in the event:
determining a time zone in the detected time information;
comparing the determined time zone to a normalized value;
generating an offset indicating the difference between the determined time zone and the normalized value;
generating a time stamp based on the detected time information and the offset; and
associating the generated time stamp with the event;
wherein the method is performed by one or more computing devices.

2 . The method of claim 1 , wherein associating the generated time stamp further comprises modifying the time information in the event to the generated time stamp.

3 . The method of claim 1 , further comprising:
in response to detecting that the time information is not present in the event:
interpolating a time stamp for the event; and
associating the interpolated time stamp with the event.

4 . The method of claim 1 , further comprising:
in response to detecting that the time information is not present in the event:
interpolating a time stamp for the event using at least one event preceding the event and at least one event succeeding the event; and
associating the interpolated time stamp with the event.

5 . The method of claim 1 , wherein the event is among a plurality of events organized from received raw data, each event in the plurality of events includes a portion of the received raw data.

6 . The method of claim 1 , further comprising:
transforming received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events.

7 . The method of claim 1 , further comprising:
transforming received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events; receiving a search query; executing the search query across the plurality of time stamped events.

8 . The method of claim 1 , further comprising:
transforming received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events; indexing the plurality of time stamped events; receiving a search query; executing the search query across the plurality of time stamped events.

9 . The method of claim 1 , further comprising:
transforming received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events; indexing the plurality of time stamped events; receiving a search query; executing the search query across the plurality of indexed time stamped events.

10 . The method of claim 1 , further comprising:
identifying a domain for the event; wherein the detecting whether time information is present in the event further comprises:
determining a location of the time information within the event using the identified domain.

11 . The method of claim 1 , wherein the event is among a plurality of events organized from received raw data, each event in the plurality of events includes a portion of the received raw data, wherein the raw data includes machine data.

12 . The method of claim 1 , wherein the event is among a plurality of events organized from received raw data, each event in the plurality of events includes a portion of the received raw data, wherein the raw data includes unstructured data.

13 . An apparatus, comprising:
an event processor, implemented at least partially in hardware, that detects whether time information is present in an event that includes a portion of raw data; in response to the event processor detecting that the time information is present in the event, the event processor:
determines a time zone in the detected time information;
compares the determined time zone to a normalized value;
generates an offset indicating the difference between the determined time zone and the normalized value;
generates a time stamp based on the detected time information and the offset; and
associates the generated time stamp with the event.

14 . The apparatus of claim 13 , wherein the event processor further:
modifies the time information in the event to the generated time stamp.

15 . The apparatus of claim 13 , wherein the event processor in response to detecting that the time information is not present in the event, further:
interpolates a time stamp for the event using at least one event preceding the event and at least one event succeeding the event; and associates the interpolated time stamp with the event.

16 . The apparatus of claim 13 , wherein the event processor further:
transforms received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events; indexes the plurality of time stamped events; receives a search query; executes the search query across the plurality of time stamped events.

17 . One or more non-transitory computer-readable storage media, storing one or more sequences of instructions, which when executed by one or more processors cause performance of:
detecting whether time information is present in an event that includes a portion of raw data; in response to detecting that the time information is present in the event:
determining a time zone in the detected time information;
comparing the determined time zone to a normalized value;
generating an offset indicating the difference between the determined time zone and the normalized value;
generating a time stamp based on the detected time information and the offset; and
associating the generated time stamp with the event.

18 . The one or more non-transitory computer-readable storage media of claim 17 , wherein associating the generated time stamp further comprises modifying the time information in the event to the generated time stamp.

19 . The one or more non-transitory computer-readable storage media of claim 17 , further comprising:
in response to detecting that the time information is not present in the event:
interpolating a time stamp for the event using at least one event preceding the event and at least one event succeeding the event; and
associating the interpolated time stamp with the event.

20 . The one or more non-transitory computer-readable storage media of claim 17 , further comprising:
transforming received raw data into a plurality of time stamped events; wherein the event is among the plurality of time stamped events; indexing the plurality of time stamped events; receiving a search query; executing the search query across the plurality of time stamped events.

Assignees

• Splunk Inc

Inventors

• Swan Erik M
• Carasso R David
• Das Robin Kumar
• Greene Rory
• Hall Bradley
• Mealy Nicholas Christian
• Murphy Brian Philip
• Sorkin Stephen Phillip
• Stechert Andre David
• Baum Michael Joseph

Classifications

• G06F16/2272 Primary

  Management thereof · CPC title

• G06F17/30864 Primary

  Physics · mapped topic

• G06F16/2477 Primary

  Temporal data queries · CPC title

• G06F7/00

  Methods or arrangements for processing data by operating upon the order or content of the data handled (logic circuits H03K19/00) · CPC title

• G06F17/00 Primary

  Digital computing or data processing equipment or methods, specially adapted for specific functions (information retrieval, database structures or file system structures therefor G06F16/00) · CPC title