System and Method for Providing Data as a Service (DaaS) in Real-time

Patent metadata

Field	Value
Publication number	US-2017061007-A1
Application number	US-201514839226-A
Country	US
Kind code	A1
Filing date	Aug 28, 2015
Priority date	Aug 28, 2015
Publication date	Mar 2, 2017
Grant date	—

Abstract

System and method embodiments are provided for implementing Data as a Service (DaaS). The system is implemented using a client side library, on a user device, and a server or proxy server to extract relevant data from relevant data sources, and process the data before returning results to the client. The client sends a user query to the proxy server, which then sends sub-queries and receive responses from multiple data sources in real-time or near real-time. The system also uses a data model that handles varying data reliability or accuracy levels in heterogeneous data sources and indicates the confidence levels in the data provided to a user or client application. The data model assigns different confidence levels for various data to distinguish between high quality data and low quality data. Thus, users are provided with more information from multiple sources without diluting high quality data with low quality data.

First claim

What is claimed is:

1 . A method for providing data as a service (DaaS) in near or real-time comprising:
receiving, at a proxy server, a query from a client application; requesting, by the proxy server from a plurality of data sources, data to service the query, wherein the data sources are heterogeneous in term of data quality, or data structure, or both data quality and data structure; obtaining the data from each data source in near or real-time; assigning a confidence level to the data from each data source in accordance with reliability of the data source; joining the data including the confidence level from the data sources; and sending the joined data back to the client application.

2 . The method of claim 1 further comprising:
collecting the data from the data sources into virtual tables using Table Valued User Defined Functions (TVUDFs) embedded in the query; and
deleting the virtual tables created by executing the TVUDFs upon completing execution of the query.

3 . The method of claim 1 , wherein the query is received as a Structured Query Language (SQL) query with a Table Valued User Defined Function (TVUDF).

4 . The method of claim 1 , wherein the data is in a form of a three-attributes including a key identifying the data, a value of the data, and a probability value indicating the confidence level.

5 . The method of claim 1 , wherein the data is obtained and sent back to the client application without persistent storage of the data at a data warehouse.

6 . The method of claim 1 , wherein the data is requested from the data sources via a plurality of corresponding data engines for handling the data quality or data structure of the data sources in real-time.

7 . The method of claim 6 , wherein at least some of the data engines exchange between each other at least some of the data to service the request.

8 . The method of claim 1 further comprising:
forwarding the query in native query language of each data source to a plurality of data engines corresponding to the data sources; and
obtaining, in near or real-time, by the data engines from the data sources, data in response to the query, wherein the data is joined at the proxy server.

9 . The method of claim 8 , wherein the server proxy is implemented in a cloud based computing platform, and wherein the client application is implemented on a user device.

10 . A method for providing data as a service (DaaS) in near or real-time comprising:
sending, by a client application on a user device a query to a proxy server; and receiving from the proxy server, in near or real-time, a joined response from a plurality of data sources, wherein the joined response includes responses from the data sources with confidence levels associated with the responses in accordance with reliability of the corresponding data sources.

11 . The method of claim 10 further comprising:
indicating with the sent query a desired quality of response; and
receiving a response from one or more data sources with corresponding confidence levels that meet the desired quality of response.

12 . The method of claim 10 , wherein in absence of an indication for quality of response by the client application, the received joined response includes responses with varying confidence levels from the data sources with varying reliability of data.

13 . The method of claim 10 , wherein the data sources are heterogeneous in term of data quality or data structure.

14 . The method of claim 10 , wherein the query is sent using Structured Query Language (SQL) query and a table value user defined function (TVUDF).

15 . The method of claim 10 , wherein the data is obtained and returned to the client application without persistent storage of the data at a data warehouse.

16 . A network server for providing data as a service (DaaS) in near or real-time, the network server comprising:
a processor; and a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to: receive a query from a client application; request, from a plurality of data sources, data to service the query, wherein the data sources are heterogeneous in term of data quality or data structure; obtain the data from each data source in near or real-time; assign a confidence level to the data from each data source in accordance with reliability of the data source; join the data including the confidence level from the data sources; and send the joined data to the client application.

17 . The network server of claim 16 , wherein the programming includes further instructions to:
collect the data from the data sources into virtual tables using Table Valued User Defined Functions (TVUDFs) embedded in the query; and delete the virtual tables created by executing the TVUDFs upon completing execution of the query.

18 . The network server of claim 16 , wherein the programming includes further instructions to:
forward, to a plurality of data engines corresponding to a plurality of data sources, the query in native query language of each data source; and obtain, in near or real-time, by the data engines of the data sources, the data in response to the query, wherein the data engines handle the data structure of the corresponding data sources in real-time.

19 . The network server of claim 16 , wherein the client application runs on a user device, and wherein the network server communicates with the user device through a cloud based computing platform.

20 . A user device for providing data as a service (DaaS) in near or real-time, the user device comprising:
a processor; and a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to: send to a server proxy a query; and receive from the server proxy, in near or real-time, a joined response from a plurality of data sources, wherein the joined response includes responses from the data sources with confidence levels associated with the responses in accordance with reliability of the corresponding data sources.

21 . The user device of claim 20 , wherein the programming includes further instructions to:
indicate with the sent query a desired quality of response; and receive a response from one or more data sources with corresponding confidence levels that meet the desired quality of response.

22 . The user device of claim 20 , wherein the programming includes further instructions to in absence of an indication for quality of response by the user device, receive the joined response including responses with varying confidence levels from the data sources with varying reliability of data.

23 . The user device of claim 20 , wherein the data sources are heterogeneous in term of data quality or data structure.

Assignees

• Futurewei Technologies Inc

Inventors

• Ezzat Ahmed Kamal

Classifications

• G06F17/30864 Primary

  Physics · mapped topic

• H04L67/28

  Electricity · mapped topic

• G06F17/30377

  Physics · mapped topic

• G06F17/30371

  Physics · mapped topic

• G06F17/30498

  Physics · mapped topic