System and method for querying hybrid multi data sources

Patent metadata

Field	Value
Publication number	US-9372891-B2
Application number	US-201314106488-A
Country	US
Kind code	B2
Filing date	Dec 13, 2013
Priority date	Dec 13, 2013
Publication date	Jun 21, 2016
Grant date	Jun 21, 2016

Abstract

A system and method of querying hybrid multi data sources includes receiving a query at a hybrid multi data source running on a computer server, creating a list of data sources, filtering the list of data sources based on data source hints included in the query, filtering the list of data sources based on data source rules, forming one or more query plans for processing the query, evaluating each of the query plans, selecting a first query plan from the query plans based on results of the evaluating, and performing the first query plan. Each query plan corresponds to a data source in the filtered list of data sources. In some examples, the first query plan has a lowest cost of each of the query plans. In some examples, the first query plan is based on probabilities associated with the cost of each of the query plans.

First claim

What is claimed is:

1. A method of processing a data source query, the method comprising:
receiving the data source query at a hybrid multi data source running on a computer server;
creating a list of data sources, each data source in the list of data sources having a data source type selected from a group consisting of a relational data base, a nosql database, a flat file, an XML file, a representational state transfer (REST) web service, and a Simple Object Access Protocol (SOAP) web service;
removing data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources and data source type hints included in the data source query;
removing data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources and data source rules identifying preferred data source types based on features of the data source query;
forming one or more query plans for processing the data source query, each query plan corresponding to a data source in the list of data sources after the removings;
evaluating each of the query plans;
selecting a first query plan from the query plans based on results of the evaluating; and
performing the first query plan.

2. The method of claim 1 , further comprising extracting the data source hints from a source type clause in the data source query.

3. The method of claim 1 , further comprising omitting the removing of data sources from the list of data sources based on the data source rules when there is no overlap between the preferred data source types and data source types of the data sources in the list of data sources.

4. The method of claim 1 wherein removing data sources from the list of data sources based on the data source rules comprises removing each of the data sources from the list of data sources that do not have a data source type in the preferred data source types.

5. The method of claim 1 , further comprising omitting the removing of data sources from the list of data sources based on the data source rules when there is an overlap between first data source types included in the data source type hints and second data source types of the data sources in the list of data sources.

6. The method of claim 1 , further comprising omitting the removing of data sources from the list of data sources based on the data source type hints when there is no overlap between first data source types included in the data source type hints and second data source types of the data sources in the list of data sources.

7. The method of claim 1 wherein removing data sources from the list of data sources based on the data source type hints comprises removing any data sources from the list of data sources that do not have a first data source type associated with second data source types included in the data source type hints.

8. The method of claim 1 wherein:
each of the query plans is associated with a respective query plan tree comprising one or more nodes; and
evaluating each of the query plans comprises adding a node cost associated with a type of each of the nodes in the respective query plan tree to a query plan cost for that query plan.

9. The method of claim 8 wherein when one of the nodes is a join node, the node cost for the join node is based on an aggregation of a cost of each sub-query plan associated with children of the join node.

10. The method of claim 1 wherein selecting the first query plan based on the results of the evaluating comprises selecting the first query plan from the query plans that has a lowest cost.

11. The method of claim 1 wherein selecting the first query plan based on the results of the evaluating comprises selecting the first query plan based on probabilities associated with costs of each of the query plans.

12. The method of claim 1 , further comprising selecting a second query plan from the query plans when the first query plan cannot be performed.

13. An application server comprising:
memory; and
one or more processors coupled to the memory;
wherein the application server hosts a hybrid multi data source comprising:
a hint resolver;
a rule evaluator coupled to the hint resolver; and
a federated query engine coupled to the hint resolver and the rule evaluator;
wherein:
the hybrid multi data source is configured to:
receive a query; and
create a list of data sources coupled to the hybrid multi data source, each data source in the list of data sources having a data source type selected from a group consisting of a relational data base, a nosql database, a flat file, an XML file, a representational state transfer (REST) web service, and a Simple Object Access Protocol (SOAP) web service;

the hint resolver is configured to:
receive the list of data sources from the hybrid multi data source; and
remove data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources and data source type hints included in the query;

the rule evaluator is configured to:
receive the list of data sources after the removings from the hybrid multi data source or the hint resolver;
receive the query from the hybrid multi data source; and
remove data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources after the removings and data source rules identifying preferred data source types based on features of the data source query;

the federated query engine is configured to:
receive the filtered list of data sources from the hint resolver or the rule evaluator;
receive the query from the hybrid multi data source;
form one or more query plans for processing the query, each query plan corresponding to a data source in the filtered list of data sources;
evaluate each of the query plans to determine a respective estimated cost to perform each of the query plans;
select a query plan from the query plans based on each respective estimated cost; and
perform the selected query plan.

14. The hybrid multi data source of claim 13 wherein the selected query plan has a lowest estimated cost from among each of the query plans.

15. The hybrid multi data source of claim 13 wherein the selected query plan is selected based on probabilities associated with estimated costs of the query plans.

16. A non-transitory machine-readable medium comprising a first plurality of machine-readable instructions which when executed by one or more processors associated with an application server are adapted to cause the one or more processors to perform a method comprising:
receiving a data query at a hybrid multi data source running on the application server;
creating a list of data sources coupled to the hybrid multi data source, each data source in the list of data sources having a data source type selected from a group consisting of a relational data base, a nosql database, a flat file, an XML file, a representational state transfer (REST) web service, and a Simple Object Access Protocol (SOAP) web service;
removing data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources and data source type hints included in the data query;
removing data sources from the list of data sources based on the data source type of each of the data sources in the list of data sources and data source rules identifying preferred data source types based on features of the data query;
forming one or more query plans for processing the data query, each query plan corr

Assignees

• Red Hat Inc

Inventors

• Eliá Hacek Over S Filip
• Nguyen Filip

Classifications

• G06F16/24542 Primary

  Plan optimisation · CPC title

• G06F16/24545

  Selectivity estimation or determination · CPC title

• G06F17/30469

  Physics · mapped topic

• G06F17/30463 Primary

  Physics · mapped topic