System and method for parallel storage transformation

Patent metadata

Field	Value
Publication number	US-10789206-B2
Application number	US-201715638330-A
Country	US
Kind code	B2
Filing date	Jun 29, 2017
Priority date	Dec 22, 2016
Publication date	Sep 29, 2020
Grant date	Sep 29, 2020

Abstract

A transformation service is provided for a plurality of transformation jobs associated with one or more storage containers. A single transformation user runs enumeration tasks associated with data stored in the one or more storage containers. When two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval.

First claim

The invention claimed is:

1. A method comprising:
performing a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
running, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.

2. The method of claim 1 further comprising:
when one of the transformation jobs is started for a storage under transformation, creating the transformation user if there are no other active transformation jobs, and granting the transformation user rights to query content for all of the storage containers necessary for transforming the one transformation job.

3. The method of claim 1 further comprising:
when one of the plurality of transformation jobs is finished for a storage under transformation, and if there are unique storage containers to transform for the one transformation job, revoking the rights to query content for all unique containers to transform for the one transformation job, wherein the transformation user maintains rights for storage containers of the one transformation job shared with other transformation jobs.

4. The method of claim 1 wherein when the transformation job is finished for a storage under transformation, and if there are no other transformation jobs that are active, the method further comprises deleting the transformation user.

5. The method of claim 1 wherein each run enumeration task results in an object with related information regarding a container identification identifying the transformation jobs concerned with the object, and the container identification and identified jobs identify target containers for the object, and wherein metadata associated with the object is read and stored in an object index a number of N times, N being the number of identified target containers.

6. The method of claim 1 wherein the transformation user runs an enumeration task unique to one or more storage containers of a first transformation job during a first time interval, the transformation user runs an enumeration task unique to one or more storage containers of a second transformation job during a second time interval, and the transformation user runs an enumeration task for one or more storage containers of both the first and second transformation jobs during a third time interval in which both the first and second transformation jobs are active.

7. A data processing system comprising:
electronic hardware circuitry constructed to:
perform a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
run, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.

8. The data processing system of claim 7 wherein the electronic hardware circuitry is further constructed to:
when one of the transformation jobs is started for a storage under transformation, create the transformation user if there are no other active transformation jobs, and grant the transformation user rights to query content for all of the storage containers necessary for transforming the one transformation job.

9. The data processing system of claim 7 wherein the electronic hardware circuitry is further constructed to:
when one of the plurality of transformation jobs is finished for a storage under transformation, and if there are unique storage containers to transform for the one transformation job, revoke the rights to query content for all unique containers to transform for the one transformation job, wherein the transformation user maintains rights for storage containers of the one transformation job shared with other transformation jobs.

10. The data processing system of claim 7 wherein when the transformation job is finished for a storage under transformation, and if there are no other transformation jobs that are active, the electronic hardware circuitry is further constructed to delete the transformation user.

11. The data processing system of claim 7 wherein each run enumeration task results in an object with related information regarding a container identification identifying the transformation jobs concerned with the object, and the container identification and identified jobs identify target containers for the object, and wherein metadata associated with the object is read and stored in an object index a number of N times, N being the number of identified target containers.

12. The data processing system of claim 7 wherein the transformation user runs an enumeration task unique to one or more storage containers of a first transformation job during a first time interval, the transformation user runs an enumeration task unique to one or more storage containers of a second transformation job during a second time interval, and the transformation user runs an enumeration task for one or more storage containers of both the first and second transformation jobs during a third time interval in which both the first and second transformation jobs are active.

13. A non-transitory computer-readable storage medium storing computer-executable instructions, the instructions causing a machine to execute a process comprising:
performing a transformation service for a plurality of transformation jobs associated with one or more storage containers; and
running, by a single transformation user, enumeration tasks associated with data stored in the one or more storage containers, wherein the single transformation user is created when a first transformation job of the plurality of transformation jobs starts, and wherein the single transformation user is granted rights to query content for the one or more storage containers necessary for transforming the plurality of transformation jobs,
wherein, when two or more of the plurality of transformation jobs are active during a time interval, the single transformation user runs an enumeration task for all of the active transformation jobs during said time interval, and wherein the enumeration task is shared among the active transformation jobs during the time interval.

14. The non-transitory computer-readable storage medium of claim 13 wherein the process further comprises:
when one of the transformation jobs is started for a storage under transformation, creating the transformation user if there are no other active transformation jobs,

Assignees

• Emc Ip Holding Co Llc

Inventors

• Danilov Mikhail
• Gutsalov Nikita
• Tchoub Ivan
• Fedorov Alexander
• Koyushev Sergey

Classifications

• G06F16/119 Primary

  Details of migration of file systems (migration mechanisms in storage systems G06F3/0647) · CPC title

• G06F16/951

  Indexing; Web crawling techniques · CPC title

• G06F16/2453

  Query optimisation · CPC title

• H04L67/06

  specially adapted for file transfer, e.g. file transfer protocol [FTP] · CPC title

• G06F16/214

  Database migration support · CPC title