System and methods for live data migration

Patent metadata

Field	Value
Publication number	US-9805071-B1
Application number	US-201715474713-A
Country	US
Kind code	B1
Filing date	Mar 30, 2017
Priority date	Nov 10, 2016
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

Systems and methods are provided for data migration. The system may comprise one or more processors and a memory storing instructions that, when executed by the one or more processors, cause the system to migrate at least one first table of a first database schema to at least one second table of a second database schema, determine a query for modifying the first table during the migration, modify the second table based at least in part on the query, and update a mutation table to describe the modification, wherein the mutation table at least describes the modification.

First claim

What is claimed is:

1. A system for live data migration, the system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to perform:
migrating at least one first table of a first database schema to at least one second table of a second database schema, the at least one first table storing live data;
determining a query for modifying the at least one first table during the migration;
modifying the at least one second table based at least in part on the query;
updating a mutation table to describe the modification, wherein the mutation table tracks one or more changes to the at least one first table;
determining at least one field to be migrated from the at least one first table to the at least one second table;
determining that no write operations have been performed to a row corresponding to the at least one field during the migration, the determining based at least in part on the mutation table; and
migrating the at least one field from the at least one first table to the at least one second table, the migrating causing data corresponding to the at least one field from the first table to be populated in one or more fields of the second table based at least in part on a respective schema of the second table.

2. The system of claim 1 , wherein the query modifies the at least one field in the at least one first table, wherein the instructions further cause the system to perform:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has been modified based at least in part on the mutation table; and
providing data corresponding to the field from the second table in response to the second query.

3. The system of claim 1 , wherein the query modifies the at least one field in the first table, wherein the instructions further cause the system to perform:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has not been modified based at least in part on the mutation table; and
providing the data corresponding to the field from the first table in response to the second query.

4. The system of claim 1 , wherein the modifying the first table includes at least one of modifying data, inserting data, or deleting data in the first table.

5. The system of claim 1 , wherein the updating the mutation table further comprises:
determining a row in the second table to which the modification was performed; and
updating the mutation table to indicate that the row has been modified.

6. The system of claim 5 , wherein the mutation table is updated to indicate the row was modified by updating a Boolean value that corresponds to the row.

7. The system of claim 1 , wherein the query deletes at least one row in the first table, wherein the instructions further cause the system to perform:
determining a second query for accessing the at least one row during the migration;
determining that the at least one row has been modified based at least in part on the mutation table;
determining that the row has been deleted from the second table; and
providing a null value in response to the second query.

8. A computer-implemented method, the method being implemented by a computing system including one or more processors and storage media storing machine-readable instructions, the method comprising:
migrating at least one first table of a first database schema to at least one second table of a second database schema, the at least one first table storing live data;
determining a query for modifying the at least one first table during the migration;
modifying the at least one second table based at least in part on the query;
updating a mutation table to describe the modification, wherein the mutation table tracks one or more changes to the at least one first table;
determining at least one field to be migrated from the at least one first table to the at least one second table;
determining that no write operations have been performed to a row corresponding to the at least one field during the migration, the determining based at least in part on the mutation table; and
migrating the at least one field from the at least one first table to the at least one second table, the migrating causing data corresponding to the at least one field from the first table to be populated in one or more fields of the second table based at least in part on a respective schema of the second table.

9. The computer-implemented method of claim 8 , wherein the query modifies the at least one field in the at least one first table, the method further comprising:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has been modified based at least in part on the mutation table; and
providing data corresponding to the field from the second table in response to the second query.

10. The computer-implemented method of claim 8 , wherein the query modifies the at least one field in the first table, the method further comprising:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has not been modified based at least in part on the mutation table; and
providing the data corresponding to the field from the first table in response to the second query.

11. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
migrating at least one first table of a first database schema to at least one second table of a second database schema, the at least one first table storing live data;
determining a query for modifying the at least one first table during the migration;
modifying the at least one second table based at least in part on the query;
updating a mutation table to describe the modification, wherein the mutation table tracks one or more changes to the at least one first table;
determining at least one field to be migrated from the at least one first table to the at least one second table;
determining that no write operations have been performed to a row corresponding to the at least one field during the migration, the determining based at least in part on the mutation table; and
migrating the at least one field from the at least one first table to the at least one second table, the migrating causing data corresponding to the at least one field from the first table to be populated in one or more fields of the second table based at least in part on a respective schema of the second table.

12. The non-transitory computer readable medium of claim 11 , wherein the query modifies the at least one field in the at least one first table, wherein the instructions further cause the processors to perform:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has been modified based at least in part on the mutation table; and
providing data corresponding to the field from the second table in response to the second query.

13. The non-transitory computer readable medium of claim 11 , wherein the query modifies the at least one field in the at least one first table, wherein the instructions further cause the processors to perform:
determining a second query for accessing the at least one field during the migration;
determining that the at least one field has not been modified based at least in part on the mutation table; and
providing the data corresponding to the field from

Assignees

• Palantir Technologies Inc

Inventors

• Ellis Joseph
• Elliot Mark
• Fink Robert
• Van Wassenhove Sandor

Classifications

• G06F16/214 Primary

  Database migration support · CPC title

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/245

  Query processing · CPC title

• G06F16/213

  with details for schema evolution support · CPC title

• G06F17/30377

  Physics · mapped topic