Concurrent automatic adaptive storage of datasets in graph databases

Patent metadata

Field	Value
Publication number	US-12277176-B2
Application number	US-202418417772-A
Country	US
Kind code	B2
Filing date	Jan 19, 2024
Priority date	Jan 16, 2018
Publication date	Apr 15, 2025
Grant date	Apr 15, 2025

Abstract

A method comprises storing data objects in a non-graph data repository, each data object including a data source record; storing metadata of the data source records in a graph data repository, comprising transforming a reference in a first data object, which links the first data object to a second data object in the non-graph data repository, into a link node having a first edge connected to a first node representing the first data object and having a second edge connected to a second node representing the second data object, in the graph data repository; receiving a query for a shortest path between two specified nodes; executing the query on the graph data repository to obtain a result set; transforming the result set into a set of data objects in the non-graph data repository; providing the set of data objects, including the corresponding data source records, in response to the query.

First claim

What is claimed is:

1. A method of managing digital entities in graph repositories and non-graph repositories, comprising:
storing data objects in a non-graph data repository, each data object including a data source record;
storing metadata of the data source records in a graph data repository, comprising transforming a reference in a first data object, which links the first data object to a second data object in the non-graph data repository, into a link node in the graph data repository having a first edge that is connected to a first node representing the first data object and having a second edge that is connected to a second node representing the second data object, in the graph data repository;
receiving a query for a shortest path between two specified nodes;
executing the query on the graph data repository to obtain a result set;
transforming the result set into a set of data objects in the non-graph data repository;
providing the set of data objects, including the corresponding data source records, to a device associated with the query,
wherein the method is performed by one or more processors.

2. The method of claim 1 , further comprising:
receiving a request to store one or more first datasets;
the storing the data objects and the metadata being performed in response to receiving the request.

3. The method of claim 1 , further comprising creating a node in the graph data repository, for each version of each data object of the data objects that conforms to a schema or ontology of the non-graph data repository.

4. The method of claim 3 , the result set including nodes corresponding to most recent visible versions of the data objects but not nodes corresponding to older versions of the data objects.

5. The method of claim 1 , further comprising transforming a property of the first data object into a property node in the graph data repository having a third edge that is connected to the first node.

6. The method of claim 1 , further comprising transforming a secure component of the first data object into a secure component node in the graph data repository having a third edge that is connected to the first node.

7. The method of claim 6 , the secure component node having a fourth edge that is connected to a fourth node in the graph data repository representing a user group.

8. The method of claim 1 , further comprising
transforming a secure component of the first data object into a secure component property of the first node,
the secure component property including an access control list.

9. The method of claim 8 ,
the result set including the first node,
the method further comprising verifying that a user account associated with the query has access to the result set based on the access control list.

10. The method of claim 1 , the non-graph data repository being a columnar store.

11. The method of claim 1 , the result set including a path formed by only nodes corresponding to the data objects in the graph data repository.

12. The method of claim 1 , the executing comprising traversing each link node in the graph data repository.

13. One or more computer-readable non-transitory storage media storing instructions which when executed cause one or more processors to perform a method of managing digital entities in graph repositories and non-graph repositories, the method comprising:
storing data objects in a non-graph data repository, each data object including a data source record;
storing metadata of the data source records in a graph data repository, comprising transforming a reference in a first data object, which links the first data object to a second data object in the non-graph data repository, into a link node in the graph data repository having a first edge that is connected to a first node representing the first data object and having a second edge that is connected to a second node representing the second data object, in the graph data repository;
receiving a query for a shortest path between two specified nodes;
executing the query on the graph data repository to obtain a result set;
transforming the result set into a set of data objects in the non-graph data repository;
providing the set of data objects, including the corresponding data source records, to a device associated with the query.

14. The one or more computer-readable non-transitory storage media of claim 13 , the method further comprising creating a node in the graph data repository, for each version of each data object of the data objects that conforms to a schema or ontology of the non-graph data repository.

15. The one or more computer-readable non-transitory storage media of claim 13 , the method further comprising transforming a property of the first data object into a property node in the graph data repository having a third edge that is connected to the first node.

16. The one or more computer-readable non-transitory storage media of claim 13 , the method further comprising transforming a secure component of the first data object into a secure component node in the graph data repository having a third edge that is connected to the first node.

17. The one or more computer-readable non-transitory storage media of claim 13 , the method further comprising
transforming a secure component of the first data object into a secure component property of the first node,
the secure component property including an access control list.

18. The one or more computer-readable non-transitory storage media of claim 17 ,
the result set including the first node,
the method further comprising verifying that a user account associated with the query has access to the result set based on the access control list.

19. The one or more computer-readable non-transitory storage media of claim 13 , the result set including a path formed by only nodes corresponding to the data objects in the graph data repository.

20. The one or more computer-readable non-transitory storage media of claim 13 , the executing comprising traversing each link node in the graph data repository.

Assignees

• Palantir Technologies Inc

Inventors

• Cohen David
• Carter Landon

Classifications

• G06F16/2452 Primary

  Query translation · CPC title

• G06F16/254

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F21/604

  Tools and structures for managing or administering access control systems · CPC title

• G06F16/90335

  Query processing · CPC title

• G06F16/9038

  Presentation of query results · CPC title