Concurrent automatic adaptive storage of datasets in graph databases

Patent metadata

Field	Value
Publication number	US-11880409-B2
Application number	US-202217694573-A
Country	US
Kind code	B2
Filing date	Mar 14, 2022
Priority date	Jan 16, 2018
Publication date	Jan 23, 2024
Grant date	Jan 23, 2024

Abstract

Techniques for improving database searches are described herein. In an embodiment, a computer-implemented data processing method comprises receiving a request to store a digital data object in a non-graph data repository, the data object comprising one or more first properties, the data object conforming to a first schema of the non-graph data repository; in response to the initiating, concurrently: digitally storing the data object in the non-graph data repository; transforming the data object into one or more nodes and edges of a graph; storing the one or more nodes and edges of the graph in a graph data repository that is separate from the non-graph data repository; electronically receiving a search query that specifies a path-based search; in response to receiving the search query: transforming the search query into a graph search query; submitting the graph search query to the graph data repository; receiving a result set of nodes from the graph data repository; transforming the result set of nodes into a set of corresponding data objects in the non-graph data repository; providing the set of corresponding data objects as output in a computer display.

First claim

What is claimed is:

1. A method of managing digital entities in graph repositories and non-graph repositories, comprising:
storing one or more datasets in a non-graph data repository;
transforming the one or more datasets into one or more transformed datasets, including obtaining metadata for the one or more datasets;
storing the one or more transformed datasets in a graph data repository that is separate from the non-graph data repository;
electronically receiving a request to perform an analysis on a particular dataset;
initiating the analysis using both the non-graph data repository and the graph data repository;
responding to the request as responses from both the non-graph data repository and the graph data repository arrive,
the responses including a result set from the graph data repository as a set of nodes and edges and corresponding data source records from the non-graph data repository as column values,
wherein the method is performed using one or more digital electronic computers.

2. The method of claim 1 , the storing comprising transmitting the one or more datasets to the non-graph data repository for storage in columnar data store and for indexing in an index.

3. The method of claim 1 , the transforming comprising transforming properties or attributes of data objects that are represented in the one or more datasets, using a native schema or ontology of the non-graph data repository, into nodes and edges to be stored in the graph data repository.

4. The method of claim 1 , the transforming comprising:
creating or storing a node, in the graph data repository, for each version of each data object in a first dataset of the one or more datasets that conforms to a schema or ontology of the non-graph data repository;
finding an existing node in the graph data repository and updating the existing node with edges, version nodes or property nodes based on the data object, or
mapping properties or attributes of each data object to property nodes in the graph data repository that have edges linking the property nodes to object version nodes.

5. The method of claim 1 , the initiating comprising performing query translation, leading to a first query for the non-graph data repository and a second query for the graph data repository.

6. The method of claim 1 , the responding comprising returning a path showing all nodes of the graph data repository corresponding to visible versions of data objects of the non-graph data repository.

7. The method of claim 6 , the responding comprising determining that a current user account has permission to read an access control list (ACL) and view or use a secure component associated with a node in the path.

8. The method of claim 1 , the responding comprising determining whether a property node is linked to an object version node in the graph data repository.

9. The method of claim 1 , further comprising transforming the responses obtained from the graph data repository into a set of data objects that are compatible with the non-graph data repository.

10. A computer-readable, non-transitory storage medium storing computer-executable instructions, which when executed implement a method of managing digital entities in graph repositories and non-graph repositories, the method comprising:
storing one or more datasets in a non-graph data repository;
transforming the one or more datasets into one or more transformed datasets, including obtaining metadata for the one or more datasets;
storing the one or more transformed datasets in a graph data repository that is separate from the non-graph data repository;
electronically receiving a request to perform an analysis on a particular dataset;
initiating the analysis using both the non-graph data repository and the graph data repository;
responding to the request as responses from both the non-graph data repository and the graph data repository arrive,
the responses including a result set from the graph data repository as a set of nodes and edges and corresponding data source records from the non-graph data repository as column values,
wherein the method is performed using one or more digital electronic computers.

11. The computer-readable, non-transitory storage medium of claim 10 , the storing comprising transmitting the one or more datasets to the non-graph data repository for storage in columnar data store and for indexing in an index.

12. The computer-readable, non-transitory storage medium of claim 10 , the transforming comprising transforming properties or attributes of data objects that are represented in the one or more datasets, using a native schema or ontology of the non-graph data repository, into nodes and edges to be stored in the graph data repository.

13. The computer-readable, non-transitory storage medium of claim 10 , the transforming comprising:
creating or storing a node, in the graph data repository, for each version of each data object in a first dataset of the one or more datasets that conforms to a schema or ontology of the non-graph data repository;
finding an existing node in the graph data repository and updating the existing node with edges, version nodes or property nodes based on the data object, or
mapping properties or attributes of each data object to property nodes in the graph data repository that have edges linking the property nodes to object version nodes.

14. The computer-readable, non-transitory storage medium of claim 10 , the initiating comprising performing query translation, leading to a first query for the non-graph data repository and a second query for the graph data repository.

15. The computer-readable, non-transitory storage medium of claim 10 , the responding comprising returning a path showing all nodes of the graph data repository corresponding to visible versions of data objects of the non-graph data repository.

16. The computer-readable, non-transitory storage medium of claim 15 , the responding comprising determining that a current user account has permission to read an access control list (ACL) and view or use a secure component associated with a node in the path.

17. The computer-readable, non-transitory storage medium of claim 10 , the responding comprising determining whether a property node is linked to an object version node in the graph data repository.

18. The computer-readable, non-transitory storage medium of claim 10 , the method further comprising transforming the responses obtained from the graph data repository into a set of data objects that are compatible with the non-graph data repository.

Assignees

• Palantir Technologies Inc

Inventors

• Cohen David
• Carter Landon

Classifications

• G06F16/2452 Primary

  Query translation · CPC title

• G06F16/9024 Primary

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06F16/254

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F16/258

  Data format conversion from or to a database · CPC title

• G06F16/9038

  Presentation of query results · CPC title