Processing a query using transformed raw data

Patent metadata

Field	Value
Publication number	US-10854331-B2
Application number	US-201415522246-A
Country	US
Kind code	B2
Filing date	Oct 26, 2014
Priority date	Oct 26, 2014
Publication date	Dec 1, 2020
Grant date	Dec 1, 2020

Abstract

A transformation on raw data is applied to produce transformed data, where the transformation includes at least one selected from among a summary of the raw data or a transform of the raw data between different domains. In response to a query to access data, the query is processed using the transformed data.

First claim

What is claimed is:

1. A method comprising:
applying, by a system comprising a processor, a transformation on raw data to produce first transformed data for a first time window of a plurality of time windows and second transformed data for a second time window of the plurality of time windows,
wherein the transformation of the first transformed data for the first time window produces a summary of the raw data in the first time window and the transformation of the second transformed data for the second time window produces the second transformed data in different domains in the second time window, and
wherein the application of the transformation of the first transformed data and the transformation of the second transformed data retains a fraction of the raw data;
calculating an interpolated data sample that is missing from the first transformed data and the second transformed data; and
in response to a query to access data at a query time, processing, by the system, the query using the first and second transformed data in at least a subset of the plurality of time windows,
wherein the first and second transformed data is a compact representation of the raw data and is stored in non-disk-based storage devices, and
wherein the query is enabled to access the raw data, the interpolated data sample, the first transformed data, and the second transformed data absent transforming the data at the query time.

2. The method of claim 1 , wherein applying the transformation comprises transforming the raw data in a first domain to the transformed raw data in a second, different domain.

3. The method of claim 2 , wherein the first domain is a time domain, and the second domain is a frequency domain.

4. The method of claim 2 , wherein the transformation comprises at least one from among a Fourier transform and a wavelet transform.

5. The method of claim 1 , wherein the summary selected from among a sum of values of the raw data, a minimum of the values of the raw data, a maximum of the values of the raw data, a count of the values of the raw data, a product of the values of the raw data, and a model of the raw data.

6. The method of claim 1 , wherein the summary comprising an auto-regressive model or an auto-regressive moving average model.

7. The method of claim 1 , wherein the first time window of the plurality of time windows is independent of the second time window of the plurality of time windows.

8. The method of claim 1 , further comprising:
determining a state based on applying the transformation on the raw data for multiple time windows of the plurality of time windows.

9. The method of claim 1 , wherein applying the transformation comprises applying a chain of multiple transformations.

10. The method of claim 1 , wherein applying the transformation is performed to satisfy a specified error bound for an answer of the query.

11. The method of claim 10 , wherein the answer to the query is obtained by processing on the first or second transformed data and satisfies the specified error bound.

12. The method of claim 1 wherein the summary of the raw data in the first time window comprises deriving autoregressive (AR) modeling coefficients from the raw data.

13. The method of claim 1 wherein the transformation on the raw data further comprises down-sampling of the raw data.

14. The method of claim 1 wherein the transformation on raw data is a stateful transformation that preserves a state from the first time window to the second time window.

15. The method of claim 1 wherein the raw data includes monitored health metrics in a healthcare environment.

16. The method of claim 1 further comprising:
removing raw data at frequencies higher than a threshold frequency.

17. The method of claim 1 wherein the non-disk-based storage devices include solid state persistent storage.

18. A system comprising:
a network interface to receive raw data from at least one data source; and
at least one processor to:
apply a transformation on the raw data to produce first transformed data for a first time window of a plurality of time windows and a second transformed data for a second time window of the plurality of time windows,
wherein the transformation of the first transformed data for the first time window produces a summarization of the raw data in the first time window and the transformation of the second transformed data for the second time window produces the second transformed data in different domains in the second time window, and
wherein the application of the transformation of the first transformed data and the transformation of the second transformed data retains a fraction of the raw data;
calculating an interpolated data sample that is missing from the first transformed data and the second transformed data; and
in response to a query to access data at a query time, processing the query using the first and second transformed data in at least a subset of the plurality of time windows,
wherein the first and second transformed data is a compact representation of the raw data and is stored in non-disk-based storage devices, and
wherein the query is enabled to access the raw data, the interpolated data sample, the first transformed data, and the second transformed data absent transforming the data at the query time.

19. The system of claim 18 , wherein the first or second transformed data is selected based on at least one criterion selected from among a specified error bound for the query or based on an operation sought by the query.

20. An article comprising at least one non-transitory machine-readable storage medium storing instructions that upon execution cause a system to:
apply a transformation on raw data to produce first transformed data for a first time window of a plurality of time windows and second transformed data for a second time window of the plurality of time windows,
wherein the transformation of the first transformed data for the first time window produces a summary of the raw data in the first time window and the transformation of the second transformed data for the second time window produces the second transformed data in different domains in the second time window, and
wherein the application of the transformation of the first transformed data and the transformation of the second transformed data retains a fraction of the raw data;
calculating an interpolated data sample that is missing from the first transformed data and the second transformed data; and
in response to a query to access data at a query time, process the query using the first and second transformed data in at least a subset of the plurality of time windows,
wherein the first and second transformed data is a compact representation of the raw data and is stored in non-disk-based storage devices, and
wherein the query is enabled to access the raw data, the interpolated data sample, the first transformed data, and the second transformed data absent transforming the data at the query time.

Assignees

• Hewlett Packard Entpr Dev Lp

Inventors

• Cui Henggang
• Keeton Kimberly
• Roy Indrajit
• Viswanathan Krishnamurthy
• Volos Haris

Classifications

• G06F17/148

  Wavelet transforms · CPC title

• G16H40/63 Primary

  for local operation · CPC title

• G06F16/2453

  Query optimisation · CPC title

• G06F16/254

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title