Method and device for migrating data

Patent metadata

Field	Value
Publication number	US-10996858-B2
Application number	US-201715709770-A
Country	US
Kind code	B2
Filing date	Sep 20, 2017
Priority date	Sep 23, 2016
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Embodiments of the present disclosure relate to a method and device for migrating data. The method comprises identifying cold data in a primary storage system. The method further comprises, in response to determining that the cold data is in a non-compression state, obtaining the cold data from the primary storage system via a first interface, the first interface being configured for a user to access the primary storage system. The method further comprises obtaining, in response to determining the cold data is in a compression state, the cold data in the compression state from the primary storage system via a second interface that is different from the first interface. The method further comprises migrating the obtained cold data from the primary storage system to a secondary storage system.

First claim

We claim:

1. A method of migrating data, comprising:
identifying, via a primary server, cold data in a primary storage system;
in response to determining that the cold data is in a non-compressed state, obtaining, via a first interface, the cold data from a primary storage system, the first interface being configured for a user to access the primary storage system;
in response to determining that the cold data is in a compressed state, obtaining, via a second interface, the cold data in the compressed state from the primary storage system, the second interface being different from the first interface;
preprocessing, via a secondary server, uncompressed data from the primary storage system prior to migrating the uncompressed data to a secondary storage system, wherein preprocessing the uncompressed data includes, at least in part, caching the uncompressed data, deleting duplicated uncompressed data, compressing the uncompressed data, and encrypting the uncompressed data;
adding a flag to metadata associated with the obtained cold data in the compressed state;
processing, via the secondary server, the obtained cold data in the compressed state without decompressing the obtained cold data in the compressed state;
migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system;
dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state, wherein dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state occurs after the cold data in the primary storage system is identified and before deleting cold data from the primary storage system; and
recalling, in response to receiving a request from a user for access to a part of uncompressed data corresponding to compressed cold data, only a portion of the compressed data that corresponds to the part of uncompressed data requested, wherein the portion of the compressed data is associated with at least one of the plurality of logical segments from the secondary storage system.

2. The method according to claim 1 , further comprising:
in response to receiving a request for access to a part of the uncompressed data, determining a state of the uncompressed data; and
in response to determining that the uncompressed data is in a migrated state, determining, based on the request, at least one of the plurality of logical segments that corresponds to the part of the uncompressed data.

3. The method according to claim 1 , wherein the cold data is organized as a file, a data block, or a data object.

4. The method according to claim 1 , wherein identifying, via a primary server, cold data in a primary storage system is based upon, at least in part, a predefined policy.

5. The method according to claim 4 , wherein the predefined policy includes an identification mode.

6. The method according to claim 1 , further comprising:
updating, via the primary server, contents of metadata associated with the cold data in the primary storage system after migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system.

7. The method according to claim 1 , further comprising:
storing information in the primary server regarding a position of the migrated obtained cold data in the non-compressed state and the obtained cold data in the compressed state in the secondary storage system, wherein the information comprises at least a Uniform Resource Locator (URL) with a universally unique identifier (UUID).

8. The method according to claim 1 , wherein the mapping between indexes of a plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state is implemented in the form of a mapping relation table and one or more of a matrix and a vector.

9. The method according to claim 1 , wherein the secondary storage system comprises a lower access performance in comparison to the primary storage system.

10. The method according to claim 1 , wherein the secondary storage system further comprises a larger storage space in comparison to the primary storage system.

11. The method according to claim 1 , wherein the primary storage system is an enterprise-level storage system and the secondary storage system is a cloud storage system.

12. An electronic device, comprising:
at least one processor; and
at least one memory coupled to the at least one processor and storing instructions to be executed by the at least one processor, the instructions, when executed by the at least one processor, causing the device to perform acts including:
identifying, via a primary server, cold data in a primary storage system;
in response to determining that the cold data is in a non-compressed state, obtaining, via a first interface, the cold data from a primary storage system, the first interface being configured for a user to access the primary storage system;

in response to determining that the cold data is in a compressed state, obtaining, via a second interface, the cold data in the compressed state from the primary storage system, the second interface being different from the first interface;
preprocessing, via a secondary server, uncompressed data from the primary storage system prior to migrating the uncompressed data to a secondary storage system, wherein preprocessing the uncompressed data includes, at least in part, caching the uncompressed data, deleting duplicated uncompressed data, compressing the uncompressed data, and encrypting the uncompressed data;
adding a flag to metadata associated with the obtained cold data in the compressed state;
processing, via the secondary server, the obtained cold data in the compressed state without decompressing the obtained cold data in the compressed state;
migrating the obtained cold data in the non-compressed state and the obtained cold data in the compressed state from the primary storage system to a secondary storage system;
dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state, wherein dividing uncompressed data corresponding to the cold data in the compressed state into a plurality of logical segments and creating a mapping between indexes of the plurality of logical segments and one or more offsets of the plurality of logical segments in the cold data in the compressed state occurs after the cold data in the primary storage system is identified and before deleting cold data from the primary storage system; and
recalling, in response to receiving a request from a user for access to a part of uncompressed data corresponding to compressed cold data, only a portion of the compressed data that corresponds to the part of uncompressed data requested, wherein the portion of the compressed data is associated with at least one of the plurality of logical segments from the secondary storage system.

13. The device according to claim 12

Assignees

• Emc Ip Holding Co Llc

Inventors

• Zhao Junping
• Zhang Sen

Classifications

• G06F3/061 Primary

  Improving I/O performance · CPC title

• G06F3/0647

  Migration mechanisms · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/0604 Primary

  Improving or facilitating administration, e.g. storage management · CPC title