Performance- and cost-efficient archiving of small objects

Patent metadata

Field	Value
Publication number	US-11436189-B2
Application number	US-201916279916-A
Country	US
Kind code	B2
Filing date	Feb 19, 2019
Priority date	Feb 19, 2019
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

A method includes receiving a plurality of files where each file represents an object. The method further combines selected files of the plurality of files into an aggregation file and stores the aggregation file on low-latency storage media, such as disk storage media. The method represents the aggregation file with a single inode that is stored on the low-latency storage media. The method migrates the aggregate file to high-latency storage media, such as tape storage media. By collecting files into larger aggregation files, the number of files archived is significantly reduced and the files archived are of larger size. The reduced number of files and increased size of the files improves archival and retrieval performance due to reduced metadata operation overhead. A corresponding system and computer program product are also disclosed.

First claim

The invention claimed is:

1. A method for archiving small objects in a performance- and cost-efficient manner, the method comprising:
receiving a plurality of files, each file representing an object;
combining selected files of the plurality of files into an aggregation file;
storing the aggregation file on low-latency storage media;
representing the aggregation file with a single inode stored on the low-latency storage media;
migrating the aggregation file to high-latency storage media while leaving the single inode representing the aggregation file on the low-latency storage media;
recalling the aggregation file from the high-latency storage media; and
upon recalling the aggregation file from the high-latency storage media, replacing, on the low-latency storage media, the single inode with individual inodes associated with the selected files.

2. The method of claim 1 , wherein the selected files are associated with objects in a container.

3. The method of claim 1 , wherein the selected files are at least one of: files that were created at or near the same time; and files that are likely to be accessed at or near the same time.

4. The method of claim 1 , further comprising storing, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.

5. The method of claim 1 , further comprising, upon recalling the aggregation file from the high-latency storage media, splitting the aggregation file into the selected files and storing the selected files on the low-latency storage media.

6. A computer program product for archiving small objects in a performance- and cost-efficient manner, the computer program product comprising a computer-readable medium having computer-usable program code embodied therein, the computer-usable program code configured to perform the following when executed by at least one processor:
receive a plurality of files, each file representing an object;
combine selected files of the plurality of files into an aggregation file;
store the aggregation file on low-latency storage media;
represent the aggregation file with a single inode stored on the low-latency storage media;
migrate the aggregation file to high-latency storage media while leaving the single inode representing the aggregation file on the low-latency storage media;
recall the aggregation file from the high-latency storage media; and
upon recalling the aggregation file from the high-latency storage media, replace, on the low-latency storage media, the single inode with individual inodes associated with the selected files.

7. The computer program product of claim 6 , wherein the selected files are associated with objects in a container.

8. The computer program product of claim 6 , wherein the selected files are at least one of: files that were created at or near the same time; and files that are likely to be accessed at or near the same time.

9. The computer program product of claim 6 , wherein the computer-usable program code is further configured to store, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.

10. The computer program product of claim 6 , wherein the computer-usable program code is further configured to, upon recalling the aggregation file from the high-latency storage media, split the aggregation file into the selected files and store the selected files on the low-latency storage media.

11. A system for archiving small objects in a performance- and cost-efficient manner, the system comprising:
at least one processor;
at least one memory device coupled to the at least one processor and storing instructions for execution on the at least one processor, the instructions causing the at least one processor to:
receive a plurality of files, each file representing an object;
combine selected files of the plurality of files into an aggregation file;
store the aggregation file on low-latency storage media;
represent the aggregation file with a single inode stored on the low-latency storage media;
migrate the aggregation file to high-latency storage media while leaving the single inode representing the aggregation file on the low-latency storage media;
recall the aggregation file from the high-latency storage media; and
upon recalling the aggregation file from the high-latency storage media, replace, on the low-latency storage media, the single inode with individual inodes associated with the selected files.

12. The system of claim 11 , wherein the selected files are associated with objects in a container.

13. The system of claim 11 , wherein the instructions further cause the at least one processor to store, in at least one of the aggregation file and a location external to the aggregation file, a mapping indicating positions of the selected files within the aggregation file.

14. The system of claim 11 , wherein the instructions further cause the at least one processor to, upon recalling the aggregation file from the high-latency storage media, split the aggregation file into the selected files and store the selected files on the low-latency storage media.

Assignees

• IBM

Inventors

• Ngo Khanh V
• Sarafijanovic Slavisa
• Mueller-Wicke Dominic
• Lorenz Simon
• Seipp Harald
• Ishimoto Takeshi

Classifications

• G06F3/0685 Primary

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title

• G06F16/113 Primary

  Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446) · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F16/185

  Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof (details of archiving G06F16/11) · CPC title

• G06F3/0649

  Lifecycle management · CPC title