Optimized query execution in a distributed data stream processing environment

Patent metadata

Field	Value
Publication number	US-2016004751-A1
Application number	US-201314761452-A
Country	US
Kind code	A1
Filing date	Feb 15, 2013
Priority date	Feb 15, 2013
Publication date	Jan 7, 2016
Grant date	—

Abstract

A controller for controlling the processing of a data stream by a data stream processing system having a plurality of data stream management systems, DSMSs. Each DSMS is arranged to execute a respective continuous query to generate an output data stream having continuous query execution results. The controller controls the execution of the continuous query on a windowed portion of a data stream when different DSMSs receive different parts of the data for the windowed portion. The controller selects, for processing of the windowed portion of the data stream, a single DSMS that is to execute the respective continuous query. The controller generates a control signal to cause only the selected DSMS to execute the continuous query using data in the data stream received thereby, so that the continuous query is executed on data in the windowed portion of the data stream only by the selected DSMS.

First claim

1 . A controller for controlling the processing of a data stream by a data stream processing system comprising a plurality of data stream management systems, DSMSs, each DSMS being configured to execute a respective continuous query comprising an operator arranged to operate on windowed portions of the input data stream to generate an output data stream comprising continuous query execution results, the controller being configured to control the execution of the continuous query on a windowed portion of the data stream when different DSMSs receive different parts of the data for the windowed portion, the controller comprising:
a reception predictor configured to predict how the identity of the DSMS receiving the data stream will change based on prior receptions of the data stream by the DSMSs; a DSMS selector configured to select, for processing of the windowed portion of the data stream, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, a single DSMS of the plurality of DSMSs that is to execute the respective continuous query; and a control signal generator configured to generate a control signal to cause only the selected DSMS to execute the continuous query using data in the data stream received thereby, so that the continuous query is executed on data in the windowed portion of the data stream only by the selected DSMS.

2 . The controller of claim 1 , wherein:
the data stream processing system is configured to receive a data stream associated with a mobile communication device, the DSMS receiving the data stream during operation of the data stream processing system being dependent upon a location of the mobile communication device; and the reception predictor is configured to monitor movement of the mobile communication device and predict how the identity of the DSMS receiving the data stream will change based on the monitored movement.

3 . The controller of claim 2 , wherein:
the reception predictor further comprises a data store configured to store at least one motion pattern, each in association with respective indications of how the identity of the DSMS receiving the data stream changes as the mobile communication device follows the motion pattern; and the reception predictor is configured to compare the monitored movement with the at least one stored motion pattern, select a motion pattern from the at least one motion pattern based on the comparison, and predict how the identity of the DSMS receiving the data stream will change based on the indications associated with the selected motion pattern.

4 . The controller of claim 1 , wherein the DSMS selector is configured to:
identify, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, DSMSs that are to receive the data to be processed as a windowed portion of the data stream; determine the respective amount of data in the windowed portion of the data stream that is to be received by each of the identified DSMSs; and select the single DSMS from among the identified DSMSs based on the determined amounts of data.

5 . The controller of claim 1 , wherein the DSMS selector is configured to:
identify, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, DSMSs that are to receive the data to be processed as a windowed portion of the data stream; determine an order in which the identified DSMSs are to receive the data in the windowed portion of the data stream; and select the single DSMS from among the identified DSMSs based on the determined order.

6 . The controller of claim 4 , wherein:
the DSMS selector further comprises a Quality of Service acquisition module configured to acquire a Quality of Service, QoS, value stipulating a QoS to be achieved during execution of the continuous query in respect to an output data stream comprising the execution results of the continuous query; and the DSMS selector is configured to select, as the single DSMS that is to execute the continuous query, a DSMS of the identified DSMSs that is capable of executing the continuous query on data in the windowed portion of the data stream such that the data stream processing system provides at least the QoS stipulated by the QoS value.

7 . The controller of claim 1 , wherein the DSMS selector is further configured to monitor data processing workloads of at least a set of the DSMSs, and select the single DSMS based on the monitored data processing workloads.

8 . The controller of claim 1 , wherein:
the DSMS selector is configured to identify, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, DSMSs that are to receive data in the data stream to be processed as a windowed portion; and the control signal generator is configured to generate a control signal comprising instructions to configure each of the identified DSMSs with respective continuous queries such that only the selected DSMS executes the continuous query on data in the windowed portion of the data stream.

9 . The controller of claim 1 , wherein:
the DSMS selector is configured to identify, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, DSMSs that are to receive the data to be processed as a windowed portion of the data stream; and the control signal generator is configured to generate a control signal to cause at least one of the identified DSMSs other than the selected DSMS to forward the data from the windowed portion of the data stream received thereby to the selected DSMS.

10 . The controller of claim 8 , wherein the control signal generator is configured to generate a control signal comprising instructions to configure at least one of the identified DSMSs other than the selected DSMS with respective continuous queries each implementing a forwarding process that forwards received data from the windowed portion of the data stream to the selected DSMS for processing.

11 . A data stream processing system comprising:
a plurality of data stream management systems, DSMSs, each DSMS being configured to execute a respective continuous query comprising an operator arranged to operate on windowed portions of the input data stream to generate an output data stream comprising continuous query execution results, and during reception of the data in the data stream to be processed as a windowed portion, the DSMS receiving the data stream changes; and a controller comprising:
a reception predictor configured to predict how the identity of the DSMS receiving the data stream will change based on prior receptions of the data stream by the DSMSs;
a DSMS selector configured to select, for processing of the windowed portion of the data stream, based on the prediction made by the reception predictor and the size of a window used to obtain the windowed portion of the data stream, a single DSMS of the plurality of DSMSs that is to execute the respective continuous query; and
a control signal generator configured to generate a control signal to cause only the selected DSMS to execute the continuous query using data in the data stream received thereby, so that the continuous query is executed on data in the windowed portion of the data stream only by the selected DSMS.

12 . The controller of claim 1 , wherein the controller is included in a data stream management system apparatus.

13 . A method of controlling the processing of a data st

Assignees

• Ericsson Telefon Ab L M

Inventors

• Lafuente Alvarez Luis Maria
• Manzano Macho David

Classifications

• H04L67/10

  in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) · CPC title

• G06N5/04

  Inference or reasoning models · CPC title

• G06F16/24568 Primary

  Data stream processing; Continuous queries · CPC title

• G06F16/24542

  Plan optimisation · CPC title

• G06F17/30463

  Physics · mapped topic