Querying multi-dimensional time series data sets

Patent metadata

Field	Value
Publication number	US-12229104-B2
Application number	US-202016895447-A
Country	US
Kind code	B2
Filing date	Jun 8, 2020
Priority date	Jun 6, 2019
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

A method, performed by one or more processors, may comprise receiving a query for performing one or more computational operations on one or more multi-dimensional data sets representing multi-dimensional time series data collected in real-time from one or more sensors associated with one or more technical systems. The method may also comprise identifying the location of the one or more multi-dimensional time series data sets in one or more databases, retrieving the one or more multi-dimensional time series data sets from the identified one or more databases, and performing the one or more computational operations on the retrieved one or more multi-dimensional time series data sets. The method may also comprise generating output based on the result of the one or more computational operations indicative of one or more states of the one or more technical systems with respect to time.

First claim

What is claimed is:

1. A method, performed by one or more processors, the method comprising:
under control of a middleware analysis platform:
receiving, real-time streaming data originating from a plurality of sensors associated with one or more technical systems, the real-time streaming data representing one or more multi-dimensional time series data sets, the real-time streaming data comprising a plurality of streams associated with respective sensors and representing a dimension relating to a time-varying quantity or parameter measured or detected by the respective sensor at a plurality of time intervals, wherein the middleware analysis platform operates independently from real-time data collection of the one or more multi-dimensional time series data sets by the plurality of sensors;
prior to parsing and cleaning the real-time streaming data, storing the real-time streaming data in a cold storage as raw data received from the plurality of sensors, the raw data comprising unparsed and uncleaned data;
cleaning the real-time streaming data;
parsing a first multi-dimensional time series data set of the one or more multi-dimensional time series data sets received from a first sensor of the plurality of sensors by structuring the real-time streaming data of the first multi-dimensional time series data set according to a first format associated with a first ontology associated with the first sensor;
parsing a second multi-dimensional time series data set of the one or more multi-dimensional time series data sets received from a second sensor of the plurality of sensors by structuring the real-time streaming data of the second multi-dimensional time series data set according to a second format associated with a second ontology associated with the second sensor;
storing the parsed time series data sets in one or more time-series databases;
in response to identifying missing data or erroneous data stored in the one or more time series databases, retrieving data corresponding to the missing data or the erroneous data from the cold storage and updating the parsed time series data sets in the one or more time-series databases with said retrieved data from the cold storage;
receiving a query for performing one or more computational operations on the parsed time series data sets representing the one or more multi-dimensional time series data sets collected in real-time from the plurality of sensors associated with the one or more technical systems, and wherein the query comprises a user-defined expression comprising a plurality of operation nodes for relating the one or more multi-dimensional time series data sets with each other according to the one or more computational operations;
automatically updating the user-defined expression to reduce a quantity of operation nodes by combining two or more of the plurality of operation nodes to generate a combined operation node;
identifying a location of the one or more multi-dimensional time series data sets in one or more databases based on accessing metadata associated with the one or more multi-dimensional time series data sets in the one or more databases, said one or more databases being pre-registered with the middleware analysis platform, the metadata including identifiers of the one or more multi-dimensional time series data sets and their respective storage locations in the one or more databases;
retrieving the one or more multi-dimensional time series data sets from the one or more databases substantially in real time with receiving the query for performing the one or more computational operations; and
performing, according to the updated user-defined expression, the one or more computational operations on the retrieved one or more multi-dimensional time series data sets to generate a resultant time series data set, wherein the middleware analysis platform is configured to perform the one or more computational operations substantially in real time with receiving the query for performing the one or more computational operations;

displaying, via an interactive graphical user interface, a multi-dimensional visualization of the resultant time series data set to permit a user to analyze one or more states of the one or more technical systems in substantially real-time;
monitoring the resultant time series data set to detect a predetermined condition of the resultant time series data set, wherein the predetermined condition is based on a relationship between the first multi-dimensional time series data set and the second multi-dimensional time series data set;
in response to detecting the predetermined condition of the resultant time series data set, displaying, via the interactive graphical user interface, an alert based on the predetermined condition of the resultant time series data set, the alert comprising:
information relating to the predetermined condition of the resultant time series data set and the one or more technical systems, and
indications of one or more system operations to be performed on the one or more technical systems; and

in response to receiving one or more user selections via the interactive graphical user interface of the indications of the one or more system operations, performing one or more system operations on the one or more technical systems according to the one or more user selections.

2. The method of claim 1 , wherein the middleware analysis platform converts the received query for performing the one or more computational operations on the one or more multi-dimensional time series data sets into an expression for performing the one or more computational operations locally.

3. The method of claim 1 , wherein the multi-dimensional visualization of the resultant time series data set comprises one or more multi-dimensional graphs.

4. The method of claim 3 , wherein the one or more multi-dimensional graphs are presented, representing a sequence of time slices.

5. The method of claim 3 , wherein the one or more multi-dimensional graphs include one or more multi-dimensional scatter plots.

6. The method of claim 1 , wherein the one or more computational operations include one or more of correlation, regression and derivatives.

7. The method of claim 1 , further comprising performing an automatic operation on the one or more technical systems responsive to the predetermined condition being detected.

8. The method of claim 1 further comprising:
accessing diagnostic data comprising a historical predetermined condition previously detected, the diagnostic data comprising information relating to the historical predetermined condition and an indication of how the historical predetermined condition was resolved, wherein the historical predetermined condition comprises one or more data outliers corresponding to one or more data outliers of the resultant time series data set;
determining the information relating to the predetermined condition based on at least the information relating to the historical predetermined condition of the diagnostic data; and
determining the one or more system operations based on at least the indication of how the historical predetermined condition was resolved.

9. A computer program, stored on a non-transitory computer readable medium which, when executed by one or more processors of a data processing apparatus, causes the data processing apparatus to carry out a method comprising:
under control of a middleware analysis platform:
receiving, real-time streaming data originating from a plurality of sensors associated with one or more technical systems, the real-time streaming data representing one or more multi-dimensional time series data sets, the real-time streaming data comprising a plurality of streams associated with respective sensors

Assignees

• Palantir Technologies Inc

Inventors

• Duffield Benjamin
• Tobin David
• Dincel Hasan
• Pandya Mihir
• Barton Stephen Nicholas
• Woodward Samantha

Classifications

• G06F16/248

  Presentation of query results · CPC title

• G06F16/283

  Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP · CPC title

• G06F16/2455

  Query execution · CPC title

• G06F16/24573

  using data annotations, e.g. user-defined metadata · CPC title

• G06F16/2465

  Query processing support for facilitating data mining operations in structured databases · CPC title