Propagated deletion of database records and derived data

Patent metadata

Field	Value
Publication number	US-12229121-B2
Application number	US-202318486039-A
Country	US
Kind code	B2
Filing date	Oct 12, 2023
Priority date	Jun 12, 2017
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

A method of enabling propagated deletion in a distributed database system comprises receiving a request to delete data in a distributed database system; causing a display of a relevant dataset and a switch between applying propagated deletion or not; receiving a first selection of a subset of records from the relevant dataset using one or more filter functions and a second selection of applying propagated deletion to the subset of records; and applying propagated deletion to the subset of records to generate a new dataset.

First claim

The invention claimed is:

1. A method of enabling propagated deletion in a distributed database system, comprising:
causing a display of a relevant dataset and a switch between applying propagated deletion or not,
applying propagated deletion comprising deleting versions of a record and versions of a different record derived from the record;
receiving a first selection of a subset of records from the relevant dataset using one or more filter functions and a second selection of applying propagated deletion of the subset of records using the switch,
applying propagated deletion further comprising updating the relevant dataset and generating a new dataset from the relevant dataset.

2. The method of claim 1 , the applying comprising:
locating, based on metadata in the distributed database system, the subset of records in one or more raw datasets;
deleting the subset of records from the one or more raw datasets.

3. The method of claim 2 , the applying further comprising:
identifying, based on the metadata in the distributed database system, one or more derived datasets linked to the one or more raw datasets;
executing, in response to the deleting, one or more builds of the one or more derived datasets to obtain the new dataset.

4. The method of claim 3 , the identifying comprising loading successive derived datasets based on name or timestamp values until a relationship to a raw dataset of the one or more raw datasets is found.

5. The method of claim 3 , the executing comprising rematerializing derived tables.

6. The method of claim 3 , further comprising:
determining, based on the metadata in the distributed database system, that a derived dataset of the one or more derived datasets has been exported to a remote computer;
generating an alert of a potential need to access the remote computer.

7. The method of claim 3 , the applying further comprising deleting historical derived datasets corresponding to the one or more builds.

8. The method of claim 1 , further comprising, in response to receiving the second selection of applying propagated deletion, adding the subset of records to a deletion list as a queue.

9. The method of claim 1 , further comprising:
submitting the subset of records to a remote computer for approval;
receiving a response indicating an approval of applying propagated deletion to the subset of records.

10. The method of claim 1 , further comprising:
applying the one or more filter functions again to the relevant dataset;
verifying that the applying results in an empty set.

11. A system for enabling propagated deletion in a distributed database system, comprising:
a memory;
one or more processors coupled to the memory and configured to perform:
causing a display of a relevant dataset and a switch between applying propagated deletion or not,
applying propagated deletion comprising deleting versions of a record and versions of a different record derived from the record;
receiving a first selection of a subset of records from the relevant dataset using one or more filter functions and a second selection of applying propagated deletion of the subset of records using the switch,
applying propagated deletion further comprising updating the relevant dataset and generating a new dataset from the relevant dataset.

12. The system of claim 11 , the applying comprising:
locating, based on metadata in the distributed database system, the subset of records in one or more raw datasets;
deleting the subset of records from the one or more raw datasets.

13. The system of claim 12 , the applying further comprising:
identifying, based on the metadata in the distributed database system, one or more derived datasets linked to the one or more raw datasets;
executing, in response to the deleting, one or more builds of the one or more derived datasets to obtain the new dataset.

14. The system of claim 13 , the identifying comprising loading successive derived datasets based on name or timestamp values until a relationship to a raw dataset of the one or more raw datasets is found.

15. The system of claim 13 , the executing comprising rematerializing derived tables.

16. The system of claim 13 , the one or more processors further configured to perform:
determining, based on the metadata in the distributed database system, that a derived dataset of the one or more derived datasets has been exported to a remote computer;
generating an alert of a potential need to access the remote computer.

17. The system of claim 13 , the applying further comprising deleting historical derived datasets corresponding to the one or more builds.

18. The system of claim 11 , the one or more processors further configured to perform, in response to receiving the second selection of applying propagated deletion, adding the subset of records to a deletion list as a queue.

19. The system of claim 11 , the one or more processors further configured to perform:
submitting the subset of records to a remote computer for approval;
receiving a response indicating an approval of applying propagated deletion to the subset of records.

20. The system of claim 11 , the one or more processors further configured to perform:
applying the one or more filter functions again to the relevant dataset;
verifying that the applying results in an empty set.

Assignees

• Palantir Technologies Inc

Inventors

• Hallgren Jeppe
• Reshi Ammaar
• Thompson James

Classifications

• G06F16/254 Primary

  Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses · CPC title

• G06F16/2433

  Query languages · CPC title

• G06F16/2365

  Ensuring data consistency and integrity · CPC title

• G06F16/2386 Primary

  Bulk updating operations (data conversion details G06F16/258) · CPC title