Autonomous decision support system using configuration inflation based ETL and content modeling

Patent metadata

Field	Value
Publication number	US-9910903-B1
Application number	US-201514635099-A
Country	US
Kind code	B1
Filing date	Mar 2, 2015
Priority date	Mar 2, 2015
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

A computer-implemented method is disclosed which detects a change in a set of data sources of a DSS, such as addition of a new data source, removal of an existing data source, or a schema change of an existing data source in the set. A first set of DSS updates for pending ETL jobs of an ETL engine of the DSS are automatically determined based on the change in the set of data sources. The method automatically updates, without canceling, the pending ETL jobs of the ETL engine based on the first set of DSS updates. A second set of DSS updates for a semantic layer of a BI engine of the DSS are automatically determined based on the change in the set of data sources and based on the first set of DSS updates. The semantic layer is automatically updated based on the second set of DSS updates.

First claim

What is claimed is:

1. A computer-implemented method comprising:
detecting a change in a set of data sources of a decision support system (DSS), the change comprising addition of a new data source to the set, removal of an existing data source from the set, or a schema change of an existing data source in the set;
automatically determining a first set of DSS updates for pending extract, transform, load (ETL) jobs of an ETL engine of the DSS based on the change in the set of data sources, wherein the pending ETL jobs import data from the set of data sources into a data warehouse, and wherein automatically determining the first set of DSS updates for pending ETL lobs of the ETL engine comprises:
analyzing historical DSS updates performed for previous changes to the set of data sources;
attempting to determine first potential DSS updates using a primary artificial intelligence (AI) model; and
if the attempting identifies first potential DSS updates that have a confidence metric that is above a predefined threshold, including the first potential DSS updates in the first set of DSS updates;

automatically updating, without canceling, the pending ETL jobs of the ETL engine based on the first set of DSS updates;
automatically determining a second set of DSS updates for a semantic layer of a business intelligence (BI) engine of the DSS based on the change in the set of data sources and based on the first set of DSS updates, wherein the semantic layer comprises a mapping of fields from the data warehouse to reporting fields of the BI engine; and
automatically updating the semantic layer of the BI engine based on the second set of DSS updates.

2. The computer-implemented method of claim 1 , wherein automatically updating, without canceling, the pending ETL jobs of the ETL engine based on the first set of DSS updates comprises automatically transmitting DSS reconfiguration instructions to the ETL engine that instruct the ETL engine to update the pending ETL jobs without cancelation of the pending ETL jobs.

3. The computer-implemented method of claim 1 , wherein the ETL engine is configured to execute the pending ETL jobs in a configuration inflation mode of operation.

4. The computer-implemented method of claim 1 , further comprising:
automatically transmitting schema update instructions to the ETL engine based on the first set of DSS updates, wherein the schema update instructions instruct the ETL engine to update a schema of the data warehouse to reflect the change in the set of data sources.

5. The computer-implemented method of claim 1 , wherein automatically updating the semantic layer of the BI engine based on the second set of DSS updates comprises:
automatically invoking an Application Programming Interface (API) of the BI engine to cause the BI engine to update its semantic layer.

6. The computer-implemented method of claim 1 , wherein automatically updating the semantic layer of the BI engine yields an updated semantic layer, the method further comprising:
automatically invoking an Application Programming Interface (API) of the BI engine to cause the BI engine to update, based on the updated semantic layer, its existing report templates that comprise data from the data warehouse.

7. The computer-implemented method of claim 1 , wherein automatically determining the first set of DSS updates further comprises:
if the attempting to determine first potential DSS updates fails, or if first potential DSS updates are determined but a confidence metric of the first potential DSS updates is below a predefined threshold, using a secondary AI model to determine second potential DSS updates, and including the second potential DSS updates in the first set of DSS updates.

8. The computer-implemented method of claim 1 , wherein the data source that is added, removed, or has its schema changed is a table, and wherein automatically determining the first set of DSS updates based on the change in the set of data sources comprises:
determining a DSS update rule for the change in the set of data sources as a function of a data model of the data warehouse, a table type of the table, and the change that was detected in the table; and
determining the first set of DSS updates based on the DSS update rule.

9. The computer-implemented method of claim 1 , wherein detecting the change in the set of data sources comprises:
analyzing metadata of the set of data sources of the DSS; and
detecting the change in the set of data sources based on a change in the metadata of the set of data sources.

10. A computing device comprising:
a communication interface configured to communicate with an extract, transform, load (ETL) engine of a decision support system (DSS), a business intelligence (BI) engine of the DSS, or both the ETL engine and the BI engine; and
a processing circuit communicatively connected to the communication interface and configured to:
detect a change in a set of data sources of the DSS, the change comprising addition of a new data source to the set, removal of an existing data source from the set, or a schema change of an existing data source in the set;
automatically determine a first set of DSS updates for pending ETL jobs of the ETL engine based on the change in the set of data sources, wherein the pending ETL jobs import data from the set of data sources into a data warehouse, and wherein to automatically determine the first set of DSS updates for pending ETL jobs of the ETL engine, the processing circuit is configured to:
analyze historical DSS updates performed for previous changes to the set of data sources;
attempt to determine first potential DSS updates using a primary artificial intelligence (AI) model; and

if the attempt identifies first potential DSS updates that have a confidence metric that is above a predefined threshold, include the first potential DSS updates in the first set of DSS updates;
automatically update, without canceling, the pending ETL jobs of the ETL engine based on the first set of DSS updates;
automatically determine a second set of DSS updates for a semantic layer of the BI engine based on the change in the set of data sources and based on the first set of DSS updates, wherein the semantic layer comprises a mapping of fields from the data warehouse to reporting fields of the BI engine; and
automatically update the semantic layer of the BI engine based on the second set of DSS updates.

11. The computing device of claim 10 , wherein to automatically update, without canceling, the pending ETL jobs of the ETL engine based on the first set of DSS updates, the processing circuit is configured to automatically transmit DSS reconfiguration instructions to the ETL engine that instruct the ETL engine to update the pending ETL jobs without cancelation of the pending ETL jobs.

12. The computing device of claim 10 , wherein the ETL engine is configured to execute the pending ETL jobs in a configuration inflation mode of operation.

13. The computing device of claim 10 , wherein the processing circuit is further configured to:
automatically transmit schema update instructions to the ETL engine based on the first set of DSS updates, wherein the schema update instructions instruct the ETL engine to update a schema of the data warehouse to reflect the change in the set of data sources.

14. The computing device of claim 10 , wherein to automatically update the semantic layer of the BI engine based on the second set of DSS updates, the processing circuit is configured to:
automatically invoke an Application Programming Interface (API) of the business engine to cause the business engine to update its semantic la

Assignees

• Ca Inc

Inventors

• Jilani Abdul Khader
• Palaniappan MuthuVadugan

Classifications

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F17/30563 Primary

  Physics · mapped topic