System and methods for live data migration

Patent metadata

Field	Value
Publication number	US-11625369-B2
Application number	US-202217583751-A
Country	US
Kind code	B2
Filing date	Jan 25, 2022
Priority date	Nov 10, 2016
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.

First claim

What is claimed is:

1. A system for live data migration, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to perform:
migrating a first table of a first database schema to a second table of a second database schema, the at least one first table storing live data, the second table incorporating data from the first table and the second database schema being condensed relative to the first database schema;
receiving a query indicating a modification of a first field in the first table; and
modifying a corresponding second field in the second table based on the query indicating the modification of the first field, wherein the modifying of the second field comprises applying the modification to the second field after the migration.

2. The system of claim 1 , wherein the second database schema combines data from different columns of the first database schema into a single column.

3. The system of claim 1 , wherein the modifying of the corresponding second field in the second table comprises identifying a corresponding second row of the second table that matches a first row to which the modification of the first table was indicated.

4. The system of claim 1 , wherein the query indicates a deletion of a row in the first table.

5. The system of claim 4 , wherein the modifying of the corresponding second field comprises deleting a corresponding second row in the second table, the deletion comprising rendering the second row inaccessible.

6. The system of claim 1 , wherein the instructions further cause the system to generate a mutation table that indicates the queried modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.

7. A computer-implemented method, the method being implemented by a computing system including one or more processors and storage media storing machine-readable instructions, the method comprising:
migrating a first table of a first database schema to a second table of a second database schema, the at least one first table storing live data, the second table incorporating data from the first table and the second database schema being condensed relative to the first database schema;
receiving a query indicating a modification of a first field in the first table; and
modifying a corresponding second field in the second table based on the query indicating the modification of the first field, wherein the modifying of the corresponding second field comprises deleting the corresponding second row in the second table or rendering the second row inaccessible.

8. The computer-implemented method of claim 7 , wherein the second database schema combines data from different columns of the first database schema into a single column.

9. The computer-implemented method of claim 7 , wherein the modifying of the corresponding second field in the second table comprises identifying a corresponding second row of the second table that matches a first row to which the modification of the first table was indicated.

10. The computer-implemented method of claim 7 , wherein the query indicates a deletion of a row in the first table.

11. The computer-implemented method of claim 7 , further comprising:
generating a mutation table that indicates the queried modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.

12. The computer-implemented method of claim 11 , wherein the modifying of the second field comprises applying the modification of the second field after the migration.

13. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
migrating a first table of a first database schema to a second table of a second database schema, the at least one first table storing live data, the second table incorporating data from the first table and the second database schema being condensed relative to the first database schema;
receiving a query indicating a modification of a first field in the first table; and
modifying a corresponding second field in the second table based on the query indicating the modification of the first field, wherein the modifying of the corresponding second field comprises applying the modification to the second field after the migration.

14. The non-transitory computer readable medium of claim 13 , wherein the second database schema combines data from different columns of the first database schema into a single column.

15. The non-transitory computer readable medium of claim 13 , wherein the modifying of the corresponding second field in the second table comprises identifying a corresponding second row of the second table that matches a first row to which the modification of the first table was indicated.

16. The non-transitory computer readable medium of claim 13 , wherein the query indicates a deletion of a row in the first table.

17. The non-transitory computer readable medium of claim 16 , wherein the modifying of the corresponding second field comprises deleting a corresponding second row in the second table, the deletion comprising rendering the second row inaccessible.

18. The non-transitory computer readable medium of claim 13 , wherein the instructions further cause the one or more processors to generate a mutation table that indicates the queried modification of the first table by setting a Boolean value for a row of the mutation table corresponding to the first row the first table.

Assignees

• Palantir Technologies Inc

Inventors

• Ellis Joseph
• Elliot Mark
• Fink Robert
• Van Wassenhove Sandor

Classifications

• G06F16/214 Primary

  Database migration support · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/245

  Query processing · CPC title

• G06F16/213

  with details for schema evolution support · CPC title