Amorphous data preparation for efficient query formulation

Patent metadata

Field	Value
Publication number	US-10339133-B2
Application number	US-201314076809-A
Country	US
Kind code	B2
Filing date	Nov 11, 2013
Priority date	Nov 11, 2013
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

A method, system, and computer program product for amorphous data preparation for efficient query formulation are provided. A normalized form of representing a set of data cubes is selected. The set of data cubes includes a first data cube conforming to first data representation and a second data cube conforming to a second data representation. The normalized form selects a third data representation to represent the data of each data cube in the set. A transformation is applied to the first data cube to represent the first data cube using the third data representation to create a normalized first data cube. A set of metadata elements corresponding to the normalized first data cube is computed. The normalized first data cube and the metadata of the normalized first data cube are saved in a data store.

First claim

What is claimed is:

1. A method for amorphous data preparation, the method comprising:
selecting a normalized form of representing a set of data cubes, wherein the set of data cubes includes a first data cube from a first data source and conforming to first data representation, and a second data cube from a second data source and conforming to a second data representation, and wherein the normalized form selects a third data representation to represent the data of each data cube in the set;
applying a transformation to the first data cube to represent the first data cube using the third data representation to create a normalized first data cube;
creating, responsive to creating the normalized first data cube, a set of metadata elements corresponding to the normalized first data cube, wherein an element in the set of metadata is configured to indicate a precision of a part of a query, the precision of the part of the query being an estimate of precision achievable using a data entity from the normalized first data cube in the query;
saving the normalized first data cube and the metadata of the normalized first data cube in a data store
receiving, from the first data source, a change to the first data cube, the change comprising changed data, the changed data forming a changed first data cube;
detecting the normalized first data cube in the data store;
creating an additional normalized first data cube using the changed data, such that the additional normalized first data cube conforms to the third data representation;

saving the additional normalized first data cube in the data store such that the data store includes both the normalized first data cube and the additional normalized first data cube; and
presenting a visual representation of the normalized first data cube and a visual representation of the metadata in a visual environment for constructing the query, wherein the query uses the normalized first data cube.

2. The method of claim 1 , further comprising:
modifying the normalized first data cube, to form a modified normalized first data cube, such that the changed data is transformed to conform to the third data representation; and
saving the modified normalized first data cube in the data store.

3. The method of claim 1 , wherein the saving occurs before the transformation has normalized all data of the first data cube, forming a partially complete normalized first data cube, further comprising:
setting an element in the set of metadata elements to indicate partial completeness of the normalized first data cube; and
allowing the query to use the partially complete normalized first data cube.

4. The method of claim 1 , wherein the transformation proceeds according to a transformation rule, further comprising:
selecting the transformation rule from a set of transformation rules.

5. The method of claim 1 , wherein the set of metadata elements comprises:
an element to identify a manner of representing a portion of the first data cube in the normalized first data cube, wherein the manner of representing the portion is one of (i) as in the first data cube, (ii) modified from a value in the first data cube, and (iii) computed using a value from the first data cube.

6. The method of claim 1 , wherein the set of metadata elements comprises:
an element to indicate a level of completeness of the transformation, wherein the level of completeness corresponds to a portion of data of the first data cube that has been transformed into the normalized first data cube.

7. The method of claim 1 , wherein the set of metadata elements comprises:
an element to indicate an estimate of cost of using a data entity from the normalized first data cube in the query.

8. The method of claim 1 , further comprising:
determining that the query directed at the set of data cubes requires an output in the third data representation.

9. The method of claim 1 , further comprising:
determining that another data cube outside the set of data cubes has been normalized using the third data representation; and
determining that the other data cube outside the set of data cubes is similar to the first data cube by at least a threshold degree.

10. The method of claim 1 , further comprising:
determining that another data cube outside the set of data cubes has been normalized using the third data representation; and
determining that a cost of using the other data cube outside the set of data cubes as normalized using the third data representation is less than a cost of using the other data cube outside the set of data cubes as normalized using the second data representation.

11. The method of claim 1 , wherein the third data representation is the same as the second data representation.

12. The method of claim 1 , wherein the visual representation of the normalized first data cube comprises a graphical indication of an amount of data present in the normalized first data cube, and wherein the visual representation of the metadata comprises a graphical depiction of a cost of the query when using the normalized first data cube.

13. A computer usable program product comprising a computer usable storage device including computer usable code for amorphous data preparation, the computer usable code comprising:
computer usable code for selecting a normalized form of representing a set of data cubes, wherein the set of data cubes includes a first data cube from a first data source and conforming to first data representation, and a second data cube from a second data source and conforming to a second data representation, and wherein the normalized form selects a third data representation to represent the data of each data cube in the set;
computer usable code for applying a transformation to the first data cube to represent the first data cube using the third data representation to create a normalized first data cube;
computer usable code for creating, responsive to creating the normalized first data cube, a set of metadata elements corresponding to the normalized first data cube, wherein an element in the set of metadata is configured to indicate a precision of a part of a query, the precision of the part of the query being an estimate of precision achievable using a data entity from the normalized first data cube in the query;
computer usable code for saving the normalized first data cube and the metadata of the normalized first data cube in a data store
computer usable code for receiving, from the first data source, a change to the first data cube, the change comprising changed data, the changed data forming a changed first data cube;
computer usable code for detecting the normalized first data cube in the data store;
computer usable code for creating an additional normalized first data cube using the changed data, such that the additional normalized first data cube conforms to the third data representation;
computer usable code for saving the additional normalized first data cube in the data store such that the data store includes both the normalized first data cube and the additional normalized first data cube; and
computer usable code for presenting a visual representation of the normalized first data cube and a visual representation of the metadata in a visual environment for constructing the query, wherein the query uses the normalized first data cube.

14. The computer usable program product of claim 13 , further comprising:
computer usable code for modifying the normalized first data cube, to form a modified normalized first data cube, such that the changed data is transformed to conform to the third data representation; and
computer us

Assignees

• IBM

Inventors

• Abuelsaad Tamer E
• Boss Gregory Jensen
• Trim Craig Matthew
• Wong Albert Tien-Yuen

Classifications

• G06F16/242 Primary

  Query formulation · CPC title

• G06F16/283

  Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP · CPC title

• G06F16/2462

  Approximate or statistical queries · CPC title