Aggregate data streams in relational database systems

Patent metadata

Field	Value
Publication number	US-9824121-B2
Application number	US-201313962170-A
Country	US
Kind code	B2
Filing date	Aug 8, 2013
Priority date	Aug 10, 2012
Publication date	Nov 21, 2017
Grant date	Nov 21, 2017

Abstract

Methods, systems, and computer readable media can provide for aggregating high-rate, large-volume input data streams into low-volume output data streams in real-time. Aggregating high-rate, large-volume data streams into low-volume output data streams can be facilitated by analyzing lossless aggregation relationships among helper views within one or more continuous query tasks and executing conventional queries to derive high-level, low-volume output data streams from low-level, high-volume input data streams.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
adding one or more continuous query tasks into a registry;
removing one or more out-of-date query tasks from the registry;
parsing each respective one continuous query task of one or more continuous query tasks that are within the registry into a plurality of structured query language queries and a plurality of views associated with the plurality of structured query language queries;
defining a plurality of views comprising input views and output views for the plurality of structured query language queries, wherein one or more of the structured query language queries are associated with at least one of an input view and an output view, and wherein a plurality of input views matches a plurality of input data streams, and a plurality of output views matches a plurality of output data streams;
dividing the plurality of views into one or more view families, wherein views in each view family have a lossless aggregation relationship among the views;
merging a plurality of small view families into one or more large view families such that none of the one or more view families share a common view;
for each view family, generating a family hierarchy among the plurality of views within one or more of the view families, wherein the family hierarchy is based on matching output views and corresponding structured query language queries with input views and corresponding structured query language queries, wherein the family hierarchy comprises an operation table at the bottom of the family hierarchy, the operation table comprising input data, wherein the family hierarchy comprises one or more views that are populated directly from the operation table, the one or more views being located directly above the operation table within the family hierarchy, and wherein one or more higher level views are populated directly from the one or more views populated directly from the operation table, the one or more higher level views being located directly above the one or more views populated directly from the operation table, wherein the data populated in the one or more higher level views is of a coarser granularity than data of the one or more views that are populated directly from the operation table; and
for each view family, starting from the bottom, continuously aggregating one or more of the plurality of views into one or more aggregate output data streams, wherein the one or more aggregate output data streams comprise at least one higher level view that is populated from one or more lower level views within the associated family hierarchy.

2. The computer-implemented method of claim 1 , wherein the one or more aggregate output data streams comprise low-volume data streams.

3. The computer-implemented method of claim 1 , wherein the aggregation of the one or more of the plurality of views into the one or more aggregate output data streams occurs in real-time.

4. The computer-implemented method of claim 1 , further comprising:
presenting the one or more aggregate output data streams in one or more rolling application windows.

5. A system, comprising:
a registry operable to store query tasks, wherein one or more continuous query tasks are added to the registry, and one or more out-of-date query tasks are removed from the registry;
an engine operable to:
parse each respective one continuous query task of one or more continuous query tasks that are within the registry into one or more structured query language queries and one or more views associated with the one or more structured query language queries, wherein one or more of the structured query language queries are associated with at least one of an input view and an output view, and wherein a plurality of input views matches a plurality of input data streams, and a plurality of output views matches a plurality of output data streams;
divide the plurality of views into one or more view families, wherein views in each view family have a lossless aggregation relationship among the views;
merge a plurality of small view families into one or more large view families such that none of the one or more view families share a common view;
for each view family, generate a family hierarchy among the plurality of views within one or more of the view families, wherein the family hierarchy is based on matching output views and corresponding structured query language queries with input views and corresponding structured query language queries, wherein the family hierarchy comprises an operation table at the bottom of the family hierarchy, the operation table comprising input data, wherein the family hierarchy comprises one or more views that are populated directly from the operation table, the one or more views being located directly above the operation table within the family hierarchy, and wherein one or more higher level views are populated directly from the one or more views populated directly from the operation table, the one or more higher level views being located directly above the one or more views populated directly from the operation table, wherein the data populated in the one or more higher level views is of a coarser granularity than data of the one or more views that are populated directly from the operation table; and

a relational database management system operable to, for each view family, starting from the bottom, continously aggregate one or more input data streams into one or more output data streams, wherein the one or more output data streams comprise at least one higher level view that is populated from one or more lower level views within the associated family hierarchy.

6. The system of claim 5 , wherein the one or more input data streams comprise high-volume data streams and the one or more output data streams comprise low-volume data streams.

7. The system of claim 5 , wherein the query task comprises a query task received from an application and generated from requirements associated with the application.

8. The system of claim 5 , wherein the relational database management system is further operable to aggregate the one or more input data streams into the one or more output data streams in real-time.

9. The system of claim 5 , wherein the relational database management system is further operable to present the one or more output data streams to an application, or other destination, in one or more rolling application windows.

10. One or more non-transitory computer readable media operable to execute on a processor, the computer readable being operable to cause the processor to perform the operations comprising:
adding one or more continuous query tasks into a registry;
removing one or more out-of-date query tasks from the registry;
parsing each respective one continuous query task of one or more continuous query tasks that are within the registry into a plurality of structured query language queries and a plurality of views associated with the plurality of structured query language queries;
defining one or more input views and an output view for each of the plurality of structured query language queries, wherein one or more of the structured query language queries are associated with at least one of an input view and an output view, and wherein a plurality of input views matches a plurality of input data streams, and a plurality of output views matches a plurality of output data streams;
dividing the plurality of views into one or more view families, wherein views in each view family have a lossless aggregation relationship among the views;
merging a plurality of small view families into one or more large view families such that none of the one or more view families share a common view;
for each

Assignees

• Arris Entpr Inc
• Arris Entpr Llc

Inventors

• Wang Tong

Classifications

• G06F16/24568 Primary

  Data stream processing; Continuous queries · CPC title

• G06F17/30516 Primary

  Physics · mapped topic