Method and system for multi-core processing based time series management with pattern detection based forecasting

Patent metadata

Field	Value
Publication number	US-2019130293-A1
Application number	US-201815898109-A
Country	US
Kind code	A1
Filing date	Feb 15, 2018
Priority date	Oct 31, 2017
Publication date	May 2, 2019
Grant date	—

Abstract

System and method for multi-core processing based time series management, capable of handling high volume time series data with high speed processing for time series forecasting. The method includes storing each time series as a Structured Query Language (SQL) array in a single row using a plurality of time series parameters. Further, stored plurality of time series are analyzed on receiving a forecast request using a correlation based pattern detection mechanism. The pattern detection mechanism enables deriving a subset of forecasting models, from which an optimal or best-fit forecasting model is identified to generate one or more forecasted time series.

First claim

What is claimed is:

1 . A processor implemented method providing multi-core processing based time series management with pattern detection based forecasting, the method comprising:
receiving a plurality of time series from a plurality of data sources ( 106 ), wherein each time series among the plurality of time series is stored in a single row as a Structured Query Language (SQL) array; analyzing the stored plurality of time series on receiving a forecast request, to identify at least one of a seasonal frequency, a trend property and a seasonality property using a correlation based pattern detection mechanism, wherein the pattern detection mechanism generates a mean series, a seasonal-difference series, a seasonal-ratio series, a trend-difference series and a trend-ratio series for each time series among the plurality of time series; treating the stored plurality of time series by applying at least one of missing value corrections and outlier corrections; and deriving a subset of forecasting models from a plurality of pre-stored forecasting models based on at least one of the seasonal frequency, the trend property, the seasonality property, and the plurality of treated time series, wherein a forecasting model for time series forecasting is selected from the subset.

2 . The method of claim 1 , wherein analyzing the stored plurality of time series for identifying the seasonal frequency, the trend property and the seasonality property using the correlation based pattern detection mechanism comprises:
determining the seasonal frequency for each time series, wherein determining of the seasonal frequency comprises:
applying an Auto Correlation Function (ACF) based first order differencing to each time series among the stored plurality of time series to determine the seasonal frequency that exists for each time series; or
applying an ACF based second order differencing to each time series if seasonal frequency is not determined during the ACF based first order differencing, wherein each time series exhibits an additive property or a multiplicative property for the trend property and the seasonal property in each time series, wherein; each time series is non-stationary and strong correlations for first lags are observed for each time series being analyzed
generating the mean series from each time series based on the seasonal frequency by averaging each frequency period from initial value up to the identified seasonal frequency and repeating the pattern to extend over length of an original time series corresponding to the mean series;
deriving from each time series and the generated mean series of each time series the seasonal—difference series providing a seasonal-difference and the seasonal—ratio series providing a seasonal-ratio by subtracting and dividing respectively each points in each time series with the corresponding mean series;
identifying the seasonality property of each time series based on the derived seasonal-difference and the seasonal-ratio for each time series, wherein identifying comprises:
characterizing each time series as having the seasonality property as multiplicative when the seasonal-ratio is uniform, wherein the uniformity indicates reduced noise or no drastic variance change points observed in the series; and
characterizing each time series as having the seasonality property as additive when the seasonal difference is uniform; and

identifying a de-seasonalized series by selecting the seasonal difference or the seasonal ratio for each of the plurality of time series based on whether one of the seasonal difference and the seasonal ratio is uniform.

3 . The method of claim 2 , wherein the method further comprises:
deriving for each time series the trend-difference series by subtracting each point in the time series with the preceding point and deriving the trend-ratio series from the identified de-seasonalized series by dividing each point in the time series with the preceding point; and characterizing each time series as having the trend property as additive if the trend-difference is uniform and the trend property as multiplicative if the trend-ratio is more uniform.

4 . The method of claim 3 , wherein treating the stored plurality of time series for the missing value corrections comprises:
applying a polynomial fit on detection of the trend property to derive fitting values as interpolated trend values for missing values of each time series to generate a trend series, wherein the mean value of series is used if the trend property is not detected; de-trending each time series as per the trend property based on the derived fitting values and ignoring the parts with missing values, wherein de-trending comprises dividing the trend series from each time series if the trend property is multiplicative and subtracting the trend series from each time series if the trend property is additive; replacing the missing values by values of each of the time series that lie on forward or backward part of series lying seasonal frequency apart when seasonal property is detected, wherein mean values of up to 5 points around missing values are used if none of the forward or backward value are detected; replacing the missing values by the mean value of series if the seasonal property is not detected; multiplying or adding the resultant series, created post seasonality treatment, to the trend series depending upon the seasonal property; and repeating, recursively, the missing value correction for remaining missing values.

5 . The method of claim 2 , wherein treating the stored plurality of time series for the outlier corrections comprises:
utilizing the seasonal frequency to create the mean-series; de-seasonalizing each time series, wherein absence of seasonality property in each time series being processed indicates the time series is a de-seasonalized series, wherein de-seasonalizing comprises:
dividing the mean series from the time series if the seasonal property is multiplicative; and
subtracting the mean series from the time series if seasonality property is additive;
deriving a de-trended series from the time series by:
applying a higher order polynomial trend fit;
dividing the trend fit from the de-seasonalized series if the trend property is multiplicative; and
subtracting the trend fit from the de-seasonalized series if the trend property is additive or not detected;
applying an outlier detection mechanism on the de-trended series to detect outliers, wherein the outliers are eliminated based on user input in accordance with an acceptable preset standard deviation, to create an outlier corrected series; and re-constructing resultant series from the outlier corrected series by:
adding or multiplying trend fit to the corrected series as per trend property; and
adding or multiplying seasonal mean series to the trend corrected series as per seasonal property.

6 . The method of claim 1 , wherein the method comprises compiling a customized data from the stored plurality of time series by an aggregator component built over the SQL database on receiving a request for the customized data, wherein the compiling of the customized data comprises:
aligning a set of time series from the plurality of time series referred in the request in accordance with corresponding start date/time, wherein non-available dates/time are set to zeroes; and utilizing SQL database functions to aggregate the time series parameters based on the request to compile the customized data, wherein:
if aggregation is requested across a plurality of series along a time dimension, then, SQL functions handling ordinals of given arrays to aggregate values which are similar to aggregating columns of a matrix or custom functions are created to handle in-memory operations; and
if agg

Assignees

• Tata Consultancy Services Ltd

Inventors

• Singh Brajesh
• Shetty Sachin

Classifications

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• G06F17/30404

  Physics · mapped topic

• G06N5/047 Primary

  Pattern matching networks; Rete networks · CPC title

• G06F7/22

  Arrangements for sorting or merging computer data on continuous record carriers, e.g. tape, drum, disc · CPC title

• G06Q10/04 Primary

  Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" (market predictions or forecasting for commercial activities G06Q30/0202) · CPC title