Query-as-a-service system that provides query-result data to remote clients

Patent metadata

Field	Value
Publication number	US-11775492-B2
Application number	US-202117401222-A
Country	US
Kind code	B2
Filing date	Aug 12, 2021
Priority date	Jan 11, 2016
Publication date	Oct 3, 2023
Grant date	Oct 3, 2023

Abstract

The current document is directed to a query-as-a-service system (“QAAS system”) that collects enormous volumes of data from network-connected entities, referred to as “Things” in the phrase “Internet of Things,” persistently stores the collected data and provides a distributed-query-execution engine that allows remote clients to continuously execute queries against the collected data. In a described implementation, both the raw data and query results are persistently stored in the QAAS system, with the raw data stored for significantly longer periods of time. Query results generated by the query-processing engine are securely transmitted to QAAS remote clients for distribution to file systems, storage.

First claim

The invention claimed is:

1. A query-as-a-service system comprising:
a distributed data-streaming service that comprises a one or more computer systems, wherein the one or more computer systems include:
one or more processors; and
one or more memories that include one or more instructions that when executed by the one or more processors, cause the one or more processors to perform operations including:
receiving a first query from a first remote client computer, the first query including:
an identification of a first stream type from a plurality of stream types;
first client-specific filters, comprising one or more structured-query-language (“SQL”) queries; and
a first public encryption key;

receiving a second query from a second remote client computer, the second query including:
an identification of a second stream type from the plurality of stream types;
second client-specific filters, comprising one or more SQL queries; and
a second public encryption key;

receiving, from each of a plurality of internet-of-things (IOT) devices, one or more communications through one or more networks, each communication of the one or more communications including unstructured data associated with a function the IOT device;
aggregating the one or more communications, comprising publishing the one or more communications to a topic, wherein the topic is a category used to partition the one or more communications;
processing the aggregated communications to generate a set of processed data for use in generating one or more data streams, comprising appending, to the aggregated communications, additional data specified by the first remote client computer and the second remote client computer, wherein the additional data is obtained by modifying instrumentation code stored by one or more IoT devices of the plurality of IoT devices, wherein modifying the instrumentation code causes the one or more IoT devices to collect one or more particular data values that are not included in the unstructured data;
generating a first IOT data stream and a second IOT data stream using the set of processed data, wherein the first IOT data stream includes the first stream type and the second IOT data stream includes the second stream type;
packaging the first IOT data stream and the second IOT data stream into a plurality of communications messages for transmission by a distributor component;
filtering, based on the first client-specific filters received with the first query and the second client-specific filters received with the second query, one or more first filtered communications messages and one or more second filtered communications messages from the plurality of communications messages, wherein the filtering is based on the one or more SQL queries of the first client-specific filters and the one or more SQL queries of the second client-specific filters;
performing a first encryption, using the first public encryption key received with the first query, of the first filtered communications messages and a second encryption, using the second public encryption key received with the second query, of the second filtered communications messages;
re-encrypting, using a secure communications tunnel, the first and second encrypted filtered communications messages to generate first and second doubly encrypted communications messages;
transmitting, in response to the first query, the first doubly encrypted communications messages to the first remote client computer; and
transmitting, in response to the second query, the second doubly encrypted communications messages to the second remote client computer.

2. The system of claim 1 , wherein the operations further include:
deleting the first IOT data stream, the first filtered communications messages, the first encrypted filtered communications messages, and the first doubly encrypted communications messages, so that the one or more communications received from the plurality of IOT devices through one or more networks is stored longer than the first IOT data stream, the first filtered communications messages, the first encrypted filtered communications messages, and the first doubly encrypted communications messages.

3. The query-as-a-service system of claim 1 , wherein the IoT devices include one or more of:
network-connected processor-controlled computers;
network-connected processor-controlled devices;
network-connected processor-controlled appliances; and
network-connected devices controlled by logic circuitry.

4. The query-as-a-service system of claim 1 , wherein:
each of the one or more communications comprises event messages that include data values associated with one or more data fields;
the event messages are enriched, by the distributed data-streaming service, to include additional data values corresponding to additional fields; and
the enriched event messages are assembled into session messages by the distributed data-streaming service, each session message including data values corresponding to one or more event messages that are each associated with a particular session identifier.

5. The query-as-a-service system of claim 4 , wherein each IoT data stream is partitioned based on time into one or more time partitions, with the data streamed during a particular time partition of the one or more time partitions being stored in a mass-storage device that is associated with the particular time partition.

6. The query-as-a-service system of claim 5 , wherein the data stored in a mass-storage device is stored as separated compressed columns, each column containing the data values for a particular data field of a particular session message of the session messages, wherein the query-as-a-service system does not create and maintain indexes for the data stored in compressed columns.

7. The system of claim 1 , wherein the operations further include:
modifying the first query, by the first remote client computer, wherein the modification includes one or more changes to the first client-specific filters; and
filtering, based on the modified first client-specific filters, one or more third filtered communications messages from the plurality of communications messages.

8. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium of a computing device, including instructions configured to cause one or more data processors to perform operations including:
receiving a first query from a first remote client computer, the first query including:
an identification of a first stream type from a plurality of stream types;
first client-specific filters, comprising one or more structured-query-language (“SQL”) queries; and
a first public encryption key;

receiving a second query from a second remote client computer, the second query including:
an identification of a second stream type from the plurality of stream types;
second client-specific filters, comprising one or more SQL queries; and
a second public encryption key;

receiving, from each of a plurality of internet-of-things (IOT) devices, one or more communications through one or more networks, each communication of the one or more communications including unstructured data associated with a function the IOT device;
aggregating the one or more communications, comprising publishing the one or more communications to a topic, wherein the topic is a category used to partition the one or more communications;
processing the aggregated communications to generate a set of processed data for use in generating one or more data streams, comprising appending, to the aggregated communications, additional data specified by the first remote client computer and the second remote client computer, wherein the additional data is obtain

Assignees

• Oracle Int Corp

Inventors

• Crossley Peter
• Mcnamara Sean

Classifications

• G06F16/221 Primary

  Column-oriented storage; Management thereof · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/248

  Presentation of query results · CPC title

• G06F16/24554

  Unary operations; Data partitioning operations · CPC title

• G06F16/24561

  Intermediate data storage techniques for performance improvement · CPC title