Efficient data migration in hierarchical storage management system

Patent metadata

Field	Value
Publication number	US-2019250844-A1
Application number	US-201815894131-A
Country	US
Kind code	A1
Filing date	Feb 12, 2018
Priority date	Feb 12, 2018
Publication date	Aug 15, 2019
Grant date	—

Abstract

Embodiments provide a method, a system, and a computer program product for performing copy operations of one or more data units in a hierarchical storage management (HSM) system. The HSM system includes an upper layer and a lower layer. The upper layer includes multiple storage nodes having a grid configuration. The method comprises scheduling a copy operations of multiple data units each of which is stored in at least one of the multiple storage nodes such that loads on the copy operations are distributed among the multiple storage nodes in which the multiple data units are stored and copying the multiple data units to the lower layer in accordance with the scheduling.

First claim

What is claimed is:

1 . A computer-implemented method of managing copy operations on data units in a hierarchical storage management system, the management system comprising an upper layer and a lower layer, wherein the upper layer comprises a domain in which storage nodes are arranged for communication in a multiplexed grid configuration, wherein each data unit is stored in one or more of the storage nodes, the method comprising:
determining, by each respective storage node for each locally stored data unit therein, a smallest number of copies of each of the data units that is respectively stored in one or more of the storage nodes in the domain, wherein the determination is made based on data received over the multiplexed grid from one or more of the storage nodes respectively storing the copies; scheduling copy operations with respect to each data unit for performance by a corresponding storage node in which the smallest number of copies of a corresponding data unit is stored; and performing the copy operations in accordance with the scheduling with respect to each data unit for respective migration of each data unit to the lower layer.

2 . The method according to claim 1 , further comprising:
receiving, from a computing device connected to the management system, a write request corresponding to a data unit stored in one or more of the storage nodes, the data unit comprising an associated copy policy specifying one or more authorized storage nodes in which a copy of the data unit can be stored, wherein the copy policy further specifies a period of time during which the copy can be stored in the one or more authorized storage nodes; and executing the write request to perform a corresponding write operation on the data unit in accordance with the copy policy and with respect to the one or more authorized storage nodes.

3 . The method according to claim 1 , wherein performing the copy operations in accordance with the scheduling with respect to each data unit comprises limiting a number of times each data unit is copied from a respectively corresponding storage node to one.

4 . The method according to claim 1 , wherein the lower layer comprises storage nodes that each have a data retrieval speed and a level of availability lower than that of any of the one or more storage nodes in the upper layer.

5 . The method according to claim 1 , wherein the lower layer comprises virtualized memory resources that are abstracted to provide a memory pool.

6 . The method according to claim 5 , wherein the virtualized memory resources are abstracted by a cloud computing layer.

7 . The method according to claim 1 , wherein each storage node comprises a database connected to the multiplexed grid for communication with individual databases of each of the storage nodes for identifying one or more of the storage nodes in which a corresponding data unit is stored.

8 . The method according to claim 1 , wherein scheduling the copy operations comprises:
selecting, by a storage node in the domain, one or more of the storage nodes in the domain for the performance of the copy operations with respect to a corresponding data unit to be copied, wherein the storage nodes are selected based on a determination that a copy of the data unit to be copied is stored in one or more of the storage nodes, wherein a copy operation has not been performed on the data unit for an amount of time exceeding a predetermined threshold.

9 . The method according to claim 1 , wherein scheduling the copy operations comprises:
selecting, by a storage node in the domain, one or more of the storage nodes in the domain for the performance of the copy operations with respect to a corresponding data unit to be copied, wherein the storage nodes are selected based on a determination that the storage node itself stores a copy of the data unit to be copied, wherein a number of copies of the data unit stored in the storage node itself is less than that stored in any of the other storage nodes in the domain.

10 . A computer system for managing copy operations on data units in a hierarchical storage management system, the management system comprising an upper layer and a lower layer, wherein the upper layer comprises a domain in which storage nodes are arranged for communication in a multiplexed grid configuration, wherein each data unit is stored in one or more of the storage nodes, the computer system comprising:
one or more computer processors, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media for execution by at least one of the one or more computer processors, the program instructions, when executed by the at least one of the one or more computer processors, causing the computer system to perform a method comprising:
determining, by each respective storage node for each locally stored data unit therein, a smallest number of copies of each of the data units that is respectively stored in one or more of the storage nodes in the domain, wherein the determination is made based on data received over the multiplexed grid from one or more of the storage nodes respectively storing the copies;
scheduling copy operations with respect to each data unit for performance by a corresponding storage node in which the smallest number of copies of a corresponding data unit is stored; and
performing the copy operations in accordance with the scheduling with respect to each data unit for respective migration of each data unit to the lower layer.

11 . The computer system according to claim 10 , the method further comprising:
receiving, from a computing device connected to the management system, a write request corresponding to a data unit stored in one or more of the storage nodes, the data unit comprising an associated copy policy specifying one or more authorized storage nodes in which a copy of the data unit can be stored, wherein the copy policy further specifies a period of time during which the copy can be stored in the one or more authorized storage nodes; and executing the write request to perform a corresponding write operation on the data unit in accordance with the copy policy and with respect to the one or more authorized storage nodes.

12 . The computer system according to claim 10 , wherein performing the copy operations in accordance with the scheduling with respect to each data unit comprises limiting a number of times each data unit is copied from a respectively corresponding storage node to one.

13 . The computer system according to claim 10 , wherein the lower layer comprises storage nodes that each have a data retrieval speed and a level of availability lower than that of any of the one or more storage nodes in the upper layer.

14 . The computer system according to claim 10 , wherein the lower layer comprises virtualized memory resources that are abstracted to provide a memory pool.

15 . The computer system according to claim 14 , wherein the virtualized memory resources are abstracted by a cloud computing layer.

16 . The computer system according to claim 10 , wherein each storage node comprises a database connected to the multiplexed grid for communication with individual databases of each of the storage nodes for identifying one or more of the storage nodes in which a corresponding data unit is stored.

17 . The computer system according to claim 10 , wherein scheduling the copy operations comprises:
selecting, by a storage node in the domain, one or more of the storage nodes in the domain for the performance of the cop

Assignees

• IBM

Inventors

• Kawamura Kousei
• Masuda Koichi
• Matsui Sosuke
• Mitsuma Shinsuke
• Nohta Takeshi
• Tsuda Takahiro

Classifications

• G06F3/0613 Primary

  in relation to throughput · CPC title

• G06F3/0649 Primary

  Lifecycle management · CPC title

• G06F3/065

  Replication mechanisms · CPC title

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F2206/1012

  Load balancing · CPC title