Data migration to a cloud computing system

Patent metadata

Field	Value
Publication number	US-10216744-B2
Application number	US-201514854798-A
Country	US
Kind code	B2
Filing date	Sep 15, 2015
Priority date	May 1, 2015
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

A cloud-based migration system exposes a source-independent application programming interface for receiving data to be migrated. The data is uploaded and stored as a single entity in a cloud-based storage system. A migration system then accesses the migration package and begins migrating the data to its destination, from the cloud-based storage system.

First claim

What is claimed is:

1. A computing system, comprising:
at least one processor; and
memory storing instructions executable by the at least one processor, wherein the instructions, when executed, configure the computing system to provide:
a migration queue system configured to:
expose a source-independent application programming interface that is invoked by a migration request to migrate information, stored in source data store of a source system, to a destination system that is remote from the source system; and
queue a work item corresponding to the migration request in a work item queue;

a manifest accessing component configured to:
access a manifest of the information to be migrated; and
based on the manifest, identify content objects, from the source data store of the source system, to be migrated to the destination system; and

an import component configured to:
import the identified content objects by storing the identified content objects to a destination data store in the destination system,

wherein the manifest accessing component is configured to:
update a migration log in the manifest, the updated migration log identifying that the content objects have been stored to the destination data store.

2. The computing system of claim 1 and further comprising:
a number of virtual machines, controlled by the import component, to import the identified content objects;
wherein the instructions, when executed, configure the computing system to provide:
a bot thread controller configured to scale the number of virtual machines based on a number of work items in the work item queue.

3. The computing system of claim 1 , wherein
the migration queue system is configured to receive a migration package, that includes the manifest, in association with the migration request, the manifest identifying the information to be migrated from the source system to the destination system, and
the migration log identifies a completion status of the migration of each of the content objects.

4. The computing system of claim 1 wherein the instructions configure the computing system to provide:
an intermediate storage system configured to receive, as part of the migration request from the source system, all content objects to be migrated from the source system to the destination system;
wherein the import component is configured to import the data from the intermediate storage to the destination data store.

5. The computing system of claim 4 wherein the destination system comprises a cloud-based destination system and wherein the intermediate storage system comprises:
a cloud-based, binary large object (blob) store that stores the content objects as a binary large object.

6. The computing system of claim 1 wherein the instructions configure the computing system to provide:
a resumability system configured to track content objects that are imported to the destination system and, when a migration failure is detected, resume migration from a last-tracked content object.

7. The computing system of claim 6 wherein the manifest accessing component is configured to identify the content objects to be migrated, to the import component, in a sequential order, based on object identifiers that identify the content objects in the manifest.

8. The computing system of claim 7 wherein the instructions configure the computing system to provide:
an overwrite/ignore controller configured to control the import component to write a given identified object to the destination store, or to ignore the given identified object.

9. The computing system of claim 8 wherein the resumability system comprises:
a last object storage component configured to intermittently store a last content object identifier for a last content object identified by the manifest accessing component to be imported to the destination system.

10. The computing system of claim 9 wherein, when a migration failure is detected, the manifest accessing component is configured to begin identifying the content objects to be migrated from a beginning of the sequential order and wherein the overwrite/ignore controller is configured to control the import component to ignore all content objects identified in the sequential order prior to the last content object identified by the last content object identifier.

11. The computing system of claim 10 wherein, when the manifest accessing component identifies the last content object to the import component, the overwrite/import controller is configured to control the import component to again begin writing the identified content objects to the destination system.

12. The computing system of claim 11 wherein, once the import component again begins writing the identified content objects to the destination system, the last object storage component is configured to store a content object identifier for each identified content object, as it is identified by the manifest accessing system, as the last content object identifier.

13. The computing system of claim 12 wherein the last object storage component is configure to store a content object identifier for each identified content object, as it is identified by the manifest accessing system, as the last content object identifier, for a resumability time period, after which the last object storage system resumes intermittently storing the last content object identifier for the content objects, as they are identified by the manifest accessing system.

14. A computing system, comprising:
at least one processor; and
memory storing instructions executable by the at least one processor, wherein the instructions, when executed, configure the computing system to:
generate a migration package, in a source-independent structure, for a source of content objects to be migrated from a source system to a destination system by:
writing the content objects to the migration package in the source-independent structures; and
writing a manifest, that includes object identifiers that identify the content objects, to the migration package in the source-independent structure; and

call a source-independent application programming interface (API) on a cloud-based content migration system; and
use the source-independent API to upload the migration package, including the content objects and the manifest, to a cloud-based intermediate storage system that is remote from the source system and configured to transfer the content objects to the destination system.

15. The computing system of claim 14 wherein the migration package creation system comprises:
a metadata logging component configured to write object metadata for the content objects to a commit log, the object metadata identifying a last time the content objects were written to the migration package.

16. The computing system of claim 15 wherein the migration package creation system comprises:
a new source identifier configured to identify new content objects as objects that are added to the source system since the last time the content objects were written to the migration package; and
a differential content identifier configured to identify differential content as changes to the content objects made since the last time the content objects were written to the migration package.

17. The computing system of claim 16 wherein the migration package creation system comprises:
an incremental package generator configured to generate an incremental migration package with the new content objects and the differential content.

18. The computing system of claim 17 wherein t

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Yap Joe Keng
• Thangaraju Mahadevan
• Livingston Sean L
• Cannerozzi Roberta
• Moussa Ghania
• Estrin Ron Shimon
• Lin Yu-Ting
• Bourdages Simon
• Nguyen Trung Duc
• Cai Wenyu
• Koehne Zachary Adam
• Simek Patrick J
• Gulati Sukhvinder Singh
• Canning Ben

Classifications

• G06F16/119 Primary

  Details of migration of file systems (migration mechanisms in storage systems G06F3/0647) · CPC title

• G06F17/30079 Primary

  Physics · mapped topic

• G06F17/30368

  Physics · mapped topic

• G06F9/4856 Primary

  resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title