Method, device, and computer program product for storage management

Patent metadata

Field	Value
Publication number	US-12014070-B2
Application number	US-202117488734-A
Country	US
Kind code	B2
Filing date	Sep 29, 2021
Priority date	Sep 29, 2020
Publication date	Jun 18, 2024
Grant date	Jun 18, 2024

Abstract

Embodiments of the present disclosure provide a method, a device, and a computer program product for storage management. The method for storage management includes: acquiring a reading request for a first target file, the reading request including a first target file name of the first target file. The method further includes: determining a first target characteristic value for the first target file based on the first target file name. The method further includes: determining first target index information for the first target file from a mapping from characteristic values to index information based on the first target characteristic value, wherein the first target index information is used to index the first target file from a merged file. The method further includes: reading the first target file included in the merged file from a storage system based on the first target index information. As a result, access to a large number of small files can be converted into access to large files, thereby improving the overall file access efficiency.

First claim

The invention claimed is:

1. A method for storage management, comprising:
acquiring a reading request for a first target file, the reading request including a first target file name of the first target file;
determining a first target characteristic value for the first target file based on the first target file name, including applying a hash function to the first target file name, wherein the first target characteristic value is a hash value of the first target file name;
determining first target index information for the first target file from a mapping from characteristic values to index information based on the first target characteristic value, wherein the first target index information is used to index the first target file from a merged file which includes multiple sections for storing multiple files and is stored in a storage system, the multiple files including the first target file, and wherein the merged file is stored on a first storage device, and the first target index information is stored on a second storage device, the second storage device having a faster access speed than the first storage device; and
reading the first target file included in the merged file from the storage system based on the first target index information; acquiring a writing request for a second target file; and attaching, after the multiple sections of the merged file, a section for storing the second target file, wherein attaching a section for storing the second target file includes: determining whether the size of the second target file is lower than a threshold size; and attaching, after the multiple sections of the merged file, a section for storing the second target file if the size of the second target file is lower than the threshold size.

2. The method according to claim 1 , wherein the mapping is cached in a memory, and wherein determining the first target index information includes:
determining the first target index information by accessing the mapping from the memory.

3. The method according to claim 2 , further comprising:
reading from the storage system an index file corresponding to the merged file if it is determined that the mapping is removed from the memory, wherein the index file includes multiple entries respectively corresponding to the multiple files, each of the entries including at least a characteristic value for a corresponding file and index information for indexing the corresponding file from the merged file;
regenerating the mapping based on the characteristic values and index information included in the multiple entries in the index file; and
caching the regenerated mapping in the memory.

4. The method according to claim 3 , wherein the index file is updated asynchronously with the merged file, and the multiple entries in the index file are in the same order as that of the multiple files in the merged file, and
wherein the method further includes:
determining, by comparing the merged file with the index file, that the merged file further includes at least one file added after the multiple files,
determining at least one characteristic value respectively corresponding to the at least one file and index information for respectively indexing the at least one file from the merged file, and
creating at least one new entry in the index file for storing the determined at least one characteristic value and index information.

5. The method according to claim 1 , wherein the target index information includes an offset and a total size of a section, which is used to store the first target file, in the merged file.

6. The method according to claim 1 , wherein the multiple sections also respectively store metadata associated with the multiple files.

7. The method according to claim 1 , further comprising:
acquiring a deleting request for a third target file, the deleting request including a third target file name of the third target file;
determining a third target characteristic value for the third target file based on the third target file name;
setting, based on the third target characteristic value, a first flag bit in the mapping with regard to the third target file as deleted in order to block accesses to the third target characteristic value and third target index information that is mapped to the third target characteristic value; and
setting a second flag bit in the merged file with regard to the third target file as deleted in order to block accesses to the third target file.

8. The method according to claim 1 , further comprising:
determining whether multiple flag bits respectively corresponding to at least one of the multiple files are set as deleted;
copying the at least one file to another merged file stored in the storage system if at least one flag bit respectively corresponding to the at least one file is not set as deleted;
deleting the merged file from the storage space; and
updating the mapping based on the another merged file.

9. The method according to claim 8 , further comprising:
updating, based on the another merged file, an index file stored in the storage system and corresponding to the merged file, wherein the updated index file includes at least one entry respectively corresponding to the at least one file, each entry including at least a characteristic value of a corresponding file and index information for indexing the corresponding file from the merged file.

10. An electronic device, comprising:
at least one processor; and
at least one memory storing computer program instructions, the at least one memory and the computer program instructions being configured to cause, together with the at least one processor, the electronic device to perform actions comprising:
acquiring a reading request for a first target file, the reading request including a first target file name of the first target file;
determining a first target characteristic value for the first target file based on the first target file name, including applying a hash function to the first target file name, wherein the first target characteristic value is a hash value of the first target file name;
determining first target index information for the first target file from a mapping from characteristic values to index information based on the first target characteristic value, wherein the first target index information is used to index the first target file from a merged file which includes multiple sections for storing multiple files and is stored in a storage system, the multiple files including the first target file, and wherein the merged file is stored on a first storage device, and the first target index information is stored on a second storage device, the second storage device having a faster access speed than the first storage device; and
reading the first target file included in the merged file from the storage system based on the first target index information; wherein the actions further comprise: acquiring a writing request fora second target file; and attaching, after the multiple sections of the merged file, a section for storing the second target file, wherein attaching a section for storing the second target file includes: determining whether the size of the second target file is lower than a threshold size; and attaching, after the multiple sections of the merged file, a section for storing the second target file if the size of the second target file is lower than the threshold size.

11. The device according to claim 10 , wherein the mapping is cached in a memory, and wherein determining the first target index information includes:
determining the first target index information by accessing the mapping from the memory.

12. The devi

Assignees

• Emc Ip Holding Co Llc

Inventors

• Lv Shuo
• Chen Huan
• Zhang Lester Ming

Classifications

• G06F3/0604

  Improving or facilitating administration, e.g. storage management · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0643

  Management of files · CPC title

• G06F3/0655 Primary

  Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices · CPC title

• G06F3/0683 Primary

  Plurality of storage devices · CPC title