Multi-tenancy data analytics platform

Patent metadata

Field	Value
Publication number	US-2022012251-A1
Application number	US-202017011847-A
Country	US
Kind code	A1
Filing date	Sep 3, 2020
Priority date	Jul 9, 2020
Publication date	Jan 13, 2022
Grant date	—

Abstract

A data analytics system is disclosed that is configured to perform operations including receiving input data at a first storage location and configuring a flow service to execute a flow. The flow execution can include creating a pipeline using the flow and metadata associated with the flow, the pipeline configured to perform a data transformation specified in the flow. The flow execution can further include determining a tenancy associated with the input data using the flow. The flow execution can also include generating, using the pipeline, output data from the input data and storing, using the pipeline, the output data in a second storage location associated with the tenancy.

First claim

What is claimed is:

1 . A data analytics system comprising:
at least one processor; and at least one non-transitory computer-readable medium containing instructions that, when executed by the at least one processor cause the data system to perform operations comprising:
receiving, at a first storage location, input data;
configuring a flow service to execute a flow, flow execution comprising:
creating a pipeline using the flow and metadata associated with the flow, the pipeline configured to perform a data transformation specified in the flow;
determining a tenancy associated with the input data using the flow;
generating, using the pipeline, output data from the input data; and
storing, using the pipeline, the output data in a second storage location associated with the tenancy.

2 . The system of claim 1 , wherein determining the tenancy comprises:
extracting identifying characteristics from the input data; providing the identifying characteristics to an identity service; and receiving an indication of the tenancy from the identity service.

3 . The system of claim 2 , wherein the operations further comprise:
associating a tenancy object representing the output data with a parent object in a hierarchical data object ownership graph; and determining whether to authorize a request to display at least a portion of the output data based at least in part on the association between the tenancy object and the parent object.

4 . The system of claim 1 , wherein:
the tenancy is determined during execution of the pipeline based on values of a specified column in the input data, the column specified in the flow.

5 . The system of claim 4 , wherein:
records in the input dataset have different values of the specified column; the pipeline generates multiple output datasets, each data set corresponding to one of the multiple values; and the pipeline stores each of the multiple output datasets in locations corresponding to differing tenancies.

6 . The system of claim 1 , wherein:
configuring a flow service to execute a flow comprises obtaining metadata associated with the flow; and wherein the execution of the flow depends on the obtained metadata.

7 . The system of claim 6 , wherein:
the metadata specifies a schema of the input data or rules for associating semantics with the input data.

8 . The system of claim 1 , wherein:
configuring a flow service to execute a flow comprises obtaining an artifact implementing a data transformation; and executing the flow comprises executing the artifact.

9 . The analytics system of claim 8 , wherein:
the artifact comprises a script, executable binary, or module.

10 . The analytics system of claim 1 , wherein:
the flow comprises a JSON or YAML object.

11 . The analytics system of claim 1 , wherein:
the flow specifies that the output data can be accessed using at least one of GraphQL, SOAP, Odata, or OpenAPI.

12 . The analytics system of claim 1 , wherein:
an append-only data store includes the first storage location; and a data-lake includes the second location.

13 . A data analytics method comprising:
receiving, at a first storage location, input data; configuring a flow service to execute a flow, flow execution comprising:
obtaining metadata associated with the flow, the metadata specifying a schema of the input data or rules for associating semantics with the input data;
creating a pipeline using the flow and the metadata associated with the flow, the pipeline configured to perform a data transformation specified in the flow and execution of the flow dependent on the obtained metadata;
determining a tenancy associated with the input data using the flow, determining comprising:
extracting identifying characteristics from the input data;
providing the identifying characteristics to an identity service; and
receiving an indication of the tenancy from the identity service;

generating, using the pipeline, output data from the input data; and
storing, using the pipeline, the output data in a second storage location associated with the tenancy.

14 . The method of claim 13 , wherein the method further comprises:
associating a tenancy object representing the output data with a parent object in a hierarchical data object ownership graph; and determining whether to authorize a request to display at least a portion of the output data based at least in part on the association between the tenancy object and the parent object.

15 . The data system of claim 13 , wherein:
the tenancy is determined during execution of the pipeline based on values of a specified column in the input data, the column specified in the flow; records in the input dataset have different values of the specified column; the pipeline generates multiple output datasets, each data set corresponding to one of the multiple values; and the pipeline stores each of the multiple output datasets in locations corresponding to differing tenancies.

16 . The data system of claim 13 , wherein:
configuring a flow service to execute a flow comprises obtaining an artifact implementing a data transformation; and executing the flow comprises executing the artifact, wherein the artifact comprises a script, executable binary, or module.

17 . The data analytics system of claim 13 , wherein:
the flow comprises a JSON or YAML object.

18 . The data analytics system of claim 13 , wherein:
the flow specifies that the output data can be accessed using at least one of GraphQL, SOAP, Odata, or OpenAPI.

19 . The data analytics system of claim 13 , wherein:
an append-only data store includes the first storage location; and a data-lake includes the second location.

20 . A data analytics method comprising:
receiving, at a first storage location, input data; configuring a flow service to execute a flow, the flow comprises a JSON or YAML object and specifies that output data generated by the flow can be accessed using at least one of GraphQL, SOAP, Odata, or OpenAPI, flow execution comprising:
obtaining metadata associated with the flow, the metadata specifying a schema of the input data or rules for associating semantics with the input data;
creating a pipeline using the flow and the metadata associated with the flow, the pipeline configured to perform a data transformation specified in the flow and execution of the flow dependent on the obtained metadata;
determining a tenancy associated with the input data using the flow, determining comprising:
extracting identifying characteristics from the input data;
providing the identifying characteristics to an identity service; and
receiving an indication of the tenancy from the identity service;

generating, using the pipeline, the output data from the input data; and
storing, using the pipeline, the output data in a second storage location associated with the tenancy.

Assignees

• Fidelity Information Services Llc

Inventors

• Colcord Aaron David
• Mellott Kevin Richard
• Favela David Vincente
• Neong Jeffrey Chee-Keong

Classifications

• G06F9/3005

  to perform operations for flow control · CPC title

• G06F16/211

  Schema design and management · CPC title

• G06F16/2471

  Distributed queries · CPC title

• G06F16/2465

  Query processing support for facilitating data mining operations in structured databases · CPC title

• G06F21/6254

  by anonymising data, e.g. decorrelating personal data from the owner's identification · CPC title