Systems and methods for multi-threaded shadow migration

Patent metadata

Field	Value
Publication number	US-10102210-B2
Application number	US-201514688804-A
Country	US
Kind code	B2
Filing date	Apr 16, 2015
Priority date	Apr 18, 2014
Publication date	Oct 16, 2018
Grant date	Oct 16, 2018

Abstract

Implementations described and claimed herein provide systems and methods for migration data from a source to a target in the background using an optimal number of threads. In one implementation, a file system operation request is received at the target. The file system operation request specifying a read request for a directory having a directory level migration attribute on the target that is marked. An optimal number of threads is allocated for migrating the directory. Metadata for content in the directory is obtained from the source using the optimal number of threads. A directory entry for a file in the directory is created on the target using the metadata, the directory entry for the file associated with a file level migration attribute that is marked.

First claim

What is claimed is:

1. A method comprising:
receiving, by a computing system, an operation request from a user device, the operation request being a read request for a file in a directory of a target file system of the computing system, the directory being partially migrated from a source file system to the target file system;
determining that the file has not been migrated;
comparing a size of the file to a size threshold;
generating, based on comparing the size of the file to a size threshold, a comparison value by dividing the size of the file by the size threshold;
determining based on the comparison that multiple threads are to be used to migrate the file from the source file system to the target file system;
determining that a first portion of the file is to be migrated using a serialized technique;
determining that a second portion of the file is to be migrated using a parallelized technique;
calculating a degree of parallelism available, the degree of parallelism available being based on the first portion of the file and the second portion of the file;
determining a first number of threads based on the degree of parallelism available and a performance gain slope;
determining a second number of threads by comparing the first number of threads to the comparison value;
allocating the second number of threads;
creating a directory entry for the file in the directory on the target file system;
migrating the file from the source file system to the target file system using the second number of threads; and
processing, by the computing system, the read request.

2. The method of claim 1 , wherein the degree of parallelism available is based on a derivative of an expected performance gain of using-multiple threads.

3. The method of claim 2 , wherein the expected performance gain of using multiple threads is represented by:

G
⁡

(
n
)

=

1
+
d

1
+

d
n

,

where
⁢

⁢
d

=

p
s

,

p represents a remaining time that can be parallelized with multiple threads, s represents a time that should be serialized for correctness, and n represents a number of threads.

4. The method of claim 1 , wherein the first number of threads is equal to:

(

n
min

)

=

max
⁢

{

n
i

❘

d
*

(

d
+
1

)

(

n
i

+
d

)

2

>
Δ

}

,

where d is represented by an equation d=p/s, s represents a time that should be serialized for correctness, p represents a remaining time that can be parallelized with multiple threads, n represents a number of threads, i represents a specific thread of the n number of threads, and Δ represents the performance gain slope.

5. The method of claim 4 , wherein the second number of threads is equal to (n opt ) which is expressed as:

(

n
opt

)

=

min
⁡

[

ceil
⁡

(

val
thresh

)

,

n
min

]

,

where ceil returns a smallest integer value, val represents the size of the file, and thresh is the size threshold.

6. The method of claim 1 , wherein the second number of threads for migrating the directory is determined to be 5 or 6 threads.

7. The method of claim 1 , further comprising:
creating a space map for the file upon initiation of migration of the file.

8. The method of claim 1 , further comprising:
unmarking an attribute of the file after migration of the file is complete.

9. The method of claim 1 , wherein the second number of threads is greater than 1 where the size threshold is exceeded.

10. The method of claim 1 , wherein the size threshold is 256 MB.

11. The method of claim 1 , wherein the second number of threads is determined to be 2 threads.

12. One or more non-transitory tangible computer-readable storage media storing computer-executable instructions for performing a computer process on a computing system, the computer process comprising:
receiving, by a computing system, an operation request from a user device, the operation request being a read request for a file in a directory of a target file system of the computing system, the directory being at least partially migrated from a source file system to the target file system;
determining that the file has not been migrated;
comparing a size of

Assignees

• Oracle Int Corp

Inventors

• Nam Young Jin
• Haley Timothy Patrick
• Rao Swanand Shreekant

Classifications

• G06F16/119 Primary

  Details of migration of file systems (migration mechanisms in storage systems G06F3/0647) · CPC title

• G06F16/185

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F16/183

  Provision of network file services by network file servers, e.g. by using NFS, CIFS (network file access protocols H04L67/1097) · CPC title

• G06F16/1787

  Details of non-transparently synchronising file systems · CPC title

• G06F17/30079 Primary

  Physics · mapped topic