Method and system for continuous query processing

Patent metadata

Field	Value
Publication number	US-9652502-B2
Application number	US-201213435756-A
Country	US
Kind code	B2
Filing date	Mar 30, 2012
Priority date	Mar 30, 2012
Publication date	May 16, 2017
Grant date	May 16, 2017

Abstract

The invention relates to a method and system that provide a continuous querying mechanism that analyses real-time data streams to generate relevant results. It leverages data grid and event processing technology to provide a high performance and extremely scalable continuous querying solution. An embodiment of the invention provides a flexible querying language and data storage to allow users to register their interest in specific types of data, e.g. processes. The users are notified of matching results using an event-based mechanism. In order to efficiently analyze a large amount of data, the embodiment uses a grid infrastructure that allows storage of the data across many grid nodes and distributes the query execution workload, avoiding the bottleneck represented by constantly querying a database. Continuous queries are stored in a Continuous Query Registry and that Registry is invoked every time an insert or update event occurs on the triple store.

First claim

The invention claimed is:

1. A method of processing a continuous query on a stored data set when pertinent data in the stored data set is added or modified, the method including the steps of:
receiving a query designated as a continuous query and storing a query model representing that query containing one or more atoms which are individual queries on the data set;
when new data is added to the data set, or the data set is modified, checking the new or modified data against the atoms of the stored query model to determine if any part of the new or modified data matches an atom of the stored query model;
if, when checking the new or modified data against the atoms of the stored query model, a match is found, checking whether the new or modified data matches the stored query model as a whole;
if, when checking whether the new or modified data matches the stored query model as a whole, a match is found with the model as a whole, assigning any fixed values in the new or modified data that correspond to free variables in the query model to those free variables;
in response to assigning any fixed values in the new or modified data that correspond to free variables in the query model to those free variables, processing the query resulting from said assignment on the stored data set; and
in response to processing the query resulting from said assignment on the stored data set, returning the result of the query.

2. The method according to claim 1 wherein the data set is stored on a plurality of interconnected nodes and the processing of the query is performed by assigning the query to one of said nodes for execution by that node.

3. The method according to claim 2 wherein the node to which the query is assigned is selected at random.

4. The method according to claim 2 wherein the execution of the query by the node is performed asynchronously.

5. The method according to claim 1 wherein the method further includes the step of adding new data to the data set by bulk import.

6. The method according to claim 1 wherein the method further includes the steps of:
monitoring one or more processes;
extracting data from those processes for storage in the data set; and
storing said extracted data in the data set.

7. The method according to claim 6 wherein the processes are business processes and the data is business process data.

8. The method according to claim 1 wherein the received query further contains a designation of an observer, which is an object to which the results of the query are to be reported, and the step of returning the result sends the result of the query to the observer.

9. The method according to claim 1 wherein the received query further contains a time duration for the query, and the method further includes the step of, when the time duration has expired, stopping processing the stored query model.

10. The method according to claim 9 wherein step of stopping processing deletes the stored query model.

11. The method according to claim 1 wherein the queries are specified in SPARQL.

12. The method according to claim 1 wherein if the stored query model has a plurality of query paths, the steps of matching determine whether any of those paths do not match the new or modified data, and if any paths do not match, those paths are not executed in the step of processing.

13. The method according to claim 1 wherein the steps of checking, assigning, processing and returning are executed immediately on receipt of new data in the data set or modification of the data set.

14. The computer system for processing queries on a data set, the computer system including a processing unit having data storage and including:
one or more nodes storing part of the data in the data set and capable of executing queries on the data of the data set;
a query engine which stores a query model representing a continuous query and containing one or more atoms which are individual queries on the data set,
wherein, when new data is added to the data set, or the data set is modified:
the node to which the data is added, or on which the data is modified sends the new or modified data to the query engine;
the query engine:
checks the new or modified data against the atoms of the stored query model to determine if any part of the new or modified data matches an atom of the stored query model;
if, when checking the new or modified data against the atoms of the stored query model, a match is found, checks whether the new or modified data matches the stored query model as a whole;
if, when checking whether the new or modified data matches the stored query model as a whole, a match is found with the model as a whole, assigns any fixed values in the new or modified data that correspond to free variables in the query model to those free variables;
in response to assigning any fixed values in the new or modified data that correspond to free variables in the query model to those free variables, the computer system processes the query resulting from said assignment on the stored data set; and
in response to processing the query resulting from said assignment on the stored data set, returns the result of the query.

15. The system according to claim 14 wherein the query engine processes the query resulting from said assignment by assigning it to one of said nodes.

16. The system according to claim 15 wherein said assignment is random.

17. The system according to claim 14 wherein said assignment is asynchronous.

18. The system according to claim 14 further including a process monitor which monitors one or more processes and extracts data from said processes and stores it in said nodes.

19. The system according to claim 14 further including a data importer for adding new data to said data set by storing it in said nodes.

Assignees

• Majeed Basim
• Afzal Ali
• Leida Marcello
• Colombo Maurizio
• Khalifa Univ Of Science Tech And Res
• British Telecomm
• Emirates Telecommunications Corp

Inventors

• Majeed Basim
• Afzal Ali
• Leida Marcello
• Colombo Maurizio

Classifications

• G06F16/24568

  Data stream processing; Continuous queries · CPC title

• G06F16/80

  of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML (content-based retrieval of web data G06F16/95) · CPC title

• G06F16/24565 Primary

  Triggers; Constraints · CPC title

• G06F17/30908

  Physics · mapped topic

• G06F17/3051 Primary

  Physics · mapped topic