Efficient updating of journey instances detected within unstructured event data

Patent metadata

Field	Value
Publication number	US-11726990-B2
Application number	US-202117451300-A
Country	US
Kind code	B2
Filing date	Oct 18, 2021
Priority date	Oct 18, 2019
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

Systems and methods are disclosed for efficiently storing information identifying journey instances within unstructured event data of a data intake and processing system. Each journey instance is illustratively associated with a series of events within the unstructured event data occurring over a journey duration. Because the unstructured event data may be constantly updated, any given inspection of the event data may yield both complete and incomplete instances. Storage of instance data over time can require updating of prior incomplete journey instances with complete versions of such instance detected at a later point in time. However, a data store of the unstructured event data may be unsuited for such updating, as the store may maintain version information for deleted data to reduce possibility of data loss. To address this issue, a separate structured data store, such as a columnar time series data store, is provided to efficiently store instance information.

First claim

What is claimed is:

1. A computer-implemented method comprising:
executing, on a data store of unstructured event data, a query for instances of a journey representing a series of steps;
obtaining, in response to the query, query results representing one or more complete journey instances, each of the one or more complete journey instances representing a series of events conforming to an entirety of the series of steps; and
updating a structured data store with the query results obtained from the unstructured event data, wherein the structured data store is configured to store each journey instance as a distinct data entry within the structured data store, and wherein updating the structured data store comprises replacing an existing data entry within the structured data store representing a partial journey instance, representing at least one event conforming to a beginning of the series of steps and less than the entirety of the series of steps, with a new data entry representing a complete journey instance corresponding to the partial journey instance.

2. The computer-implemented method of claim 1 , wherein the events within the unstructured event data are handled as information not delineated by a pre-defined data structure.

3. The computer-implemented method of claim 1 , wherein entries within the structured data store are handled as information delineated by a pre-defined data structure.

4. The computer-implemented method of claim 1 , wherein entries within the structured data store are handled as information delineated as columns within a pre-defined data structure, the columns comprising one or more of beginning timestamps of journey instances, ending timestamps of journey instances, identifiers of a journey instances, or stitching identifiers of journey instances.

5. The computer-implemented method of claim 1 , wherein the query results represent journey instances associated with a time range, and wherein updating the structured data store with the query results comprises deleting past entries within the structured data store that are associated with the time range and entering the query results into the structured data store.

6. The computer-implemented method of claim 1 , wherein the method is repeated at each period of a set of periods, wherein each set of query results represent journey instances associated with a time range that is greater than a duration of each period, and wherein each set of query results partially overlaps in time with a prior set of query results.

7. The computer-implemented method of claim 1 , wherein the method is repeated at each period of a set of periods, and wherein the one or more partial journey instances are indistinguishable from the one or more complete journey instances at a current period.

8. The computer-implemented method of claim 1 , wherein the query is limited to events within the unstructured event data occurring within a specified time range.

9. The computer-implemented method of claim 1 , wherein the query is limited to events within the unstructured event data occurring within a specified time range, and wherein the specified time range is determined based on a maximum duration of the journey.

10. The computer-implemented method of claim 1 , wherein executing the query for the journey instances comprises, for each journey instance, stitching together the series of events of the instance based on a field value shared among the series of events.

11. The computer-implemented method of claim 1 , wherein the unstructured event data is held within a single-tenanted data store of a first environment, and wherein the structured data store is maintained within a multi-tenanted hosted computing environment distinct from the first environment.

12. The computer-implemented method of claim 1 , wherein the unstructured event data is held within a single-tenanted data store of a first environment, and wherein the structured data store is maintained within a multi-tenanted hosted computing environment distinct from the first environment, and wherein each data entry within the structured data store is associated with a journey identifier distinguishing instances of the journey from instances of other journeys of other tenants of the multi-tenanted hosted computing environment.

13. The computer-implemented method of claim 1 , wherein each data entry within the structured data store is associated with a time stamp and a stitching identifier of the journey instance represented by the data entry, the stitching identifier representing a field value identified within the unstructured event data as shared among at least two events of the series of events of the data entry.

14. The computer-implemented method of claim 1 , wherein the unstructured event data is held within a data store that maintains tombstone information identifying data removed from the data store, and wherein the structured data store does not maintain information identifying data removed from the data store.

15. A system comprising:
a data store including computer-executable instructions; and
a processor in communication with the data store and configured to execute the computer-executable instructions to:
execute, on a data store of unstructured event data, a query for instances of a journey representing a series of steps;
obtain, in response to the query, query results representing one or more complete journey instances, each of the one or more complete journey instances representing a series of events conforming to an entirety of the series of steps; and
update a structured data store with the query results obtained from the unstructured event data, wherein the structured data store is configured to store each journey instance as a distinct data entry within the structured data store, and wherein updating the structured data store comprises replacing an existing data entry within the structured data store representing a partial journey instance, representing at least one event conforming to a beginning of the series of steps and less than the entirety of the series of steps, with a new data entry representing a complete journey instance corresponding to the partial journey instance.

16. The system of claim 15 , wherein the query results represent journey instances associated with a time range, and wherein updating the structured data store with the query results comprises deleting past entries within the structured data store that are associated with the time range and entering the query results into the structured data store.

17. The system of claim 15 , wherein the processor is configured to execute the computer-executable instructions at each period of a set of periods, wherein each set of query results represent journey instances associated with a time range that is greater than a duration of each period, and wherein each set of query results partially overlaps in time with a prior set of query results.

18. One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by a computing system, cause the computing system to:
execute, on a data store of unstructured event data, a query for instances of a journey representing a series of steps;
obtain, in response to the query, query results representing one or more complete journey instances, each of the one or more complete journey instances representing a series of events conforming to an entirety of the series of steps; and
update a structured data store with the query results obtained from the unstructured event data, wherein the structured data store is

Assignees

• Splunk Inc

Inventors

• Dwivedi Akash
• Gupta Himanshu
• Tschetter Eric

Classifications

• G06F16/2379 Primary

  Updates performed during online database operations; commit processing · CPC title

• G06F9/54

  Interprogram communication · CPC title

• G06F16/221

  Column-oriented storage; Management thereof · CPC title

• G06F16/248

  Presentation of query results · CPC title

• G06F16/2477 Primary

  Temporal data queries · CPC title