Method, device, and computer program product for determining space consumption

Patent metadata

Field	Value
Publication number	US-12511045-B2
Application number	US-202418931273-A
Country	US
Kind code	B2
Filing date	Oct 30, 2024
Priority date	Apr 26, 2024
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

Techniques for determining space consumption involve acquiring a first feature set of a storage system, wherein the first feature set comprises at least a size of logical space of the storage system. Such techniques further involve determining space consumption for file system checks of the storage system by a machine learning model based on the first feature set. Such techniques further involve adjusting a reserved value of physical storage space of the storage system in response to the determined space consumption meeting a predetermined condition. In this way, it is possible to predict a total amount of space to be consumed by a file system checking tool based on collection of data of the storage system, and to adjust a reserved value of physical storage space of the storage system based on the predicted value, so as to ensure metadata repair of the storage system.

First claim

The invention claimed is:

1 . A method for determining space consumption, comprising:
acquiring a first feature set of a storage system, wherein the first feature set comprises at least a size of logical space of the storage system; determining space consumption for file system checks of the storage system by a machine learning model based on the first feature set; and adjusting a reserved value of physical storage space of the storage system in response to the determined space consumption meeting a predetermined condition.

2 . The method according to claim 1 , wherein determining space consumption for file system checks of the storage system comprises:
acquiring a second feature set by performing feature extraction on the first feature set based on a feature extraction strategy, wherein the number of features of the second feature set is smaller than that of the first feature set; and determining the space consumption for file system checks of the storage system by the machine learning model based on the second feature set.

3 . The method according to claim 2 , wherein the feature extraction strategy comprises:
merging sizes of logical space of different users in the storage system; merging sizes of physical space of different users in the storage system; and merging namespace object features in the storage system, wherein the namespace object features include volume, snapshot, and clone counts.

4 . The method according to claim 2 , wherein the machine learning model is a pre-trained linear regression model, wherein determining space consumption for file system checks of the storage system comprises:
acquiring a target feature associated with the space consumption for file system checks of the storage system from the second feature set; and inputting the target feature into the linear regression model to determine the space consumption for file system checks of the storage system.

5 . The method according to claim 4 , wherein a method for determining the target feature comprises:
acquiring a first training set for determining the target feature, wherein the first training set comprises at least a size of logical space in the storage system and the space consumption for file system checks of the storage system; determining a degree of correlation between the space consumption for file system checks of the storage system and the remaining features in the first training set; and determining the target feature based on the degree of correlation.

6 . The method according to claim 4 , wherein a pre-training operation for the linear regression model comprises:
acquiring a second training set for training the linear programming model, wherein the second training set comprises at least the target feature and the space consumption for file system checks of the storage system; initializing weights and bias values of the linear regression model; determining a first loss based on the second training set; and adjusting the weights and bias values in the linear regression model based on the first loss until a preset condition is met.

7 . The method according to claim 6 , further comprising:
pre-processing features in the second training set, wherein the pre-processing comprises at least data cleaning and data transformation.

8 . The method according to claim 1 , wherein the machine learning model is a pre-trained neural network model, the method further comprising:
pre-processing features in the first feature set, wherein the pre-processing comprises at least data cleaning and normalization processing; and inputting the pre-processed features into the pre-trained neural network model to determine the space consumption for file system checks of the storage system.

9 . The method according to claim 2 , wherein the machine learning model is a pre-trained neural network model, the method further comprising:
pre-processing features in the second feature set, wherein the pre-processing comprises at least data cleaning and normalization processing; and inputting the pre-processed features into the pre-trained neural network model to determine the space consumption for file system checks of the storage system.

10 . The method according to claim 8 , wherein a pre-training operation for the neural network model comprises:
acquiring a third training set for training the neural network model, wherein the third training set comprises at least the space consumption for file system checks of the storage system; determining a second loss by the neural network model based on the third training set and parameters, wherein the parameters comprise a learning rate and the number of iterations; and adjusting weights and bias values in the neural network model based on the second loss until a preset condition is met.

11 . The method according to claim 10 , further comprising:
pre-processing features in the third training set, wherein the pre-processing comprises at least data cleaning and normalization processing.

12 . An electronic device, comprising:
at least one processor; and coupled to the at least one processor and having instructions stored thereon, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising: acquiring a first feature set of a storage system, wherein the first feature set comprises at least a size of logical space of the storage system; determining space consumption for file system checks of the storage system by a machine learning model based on the first feature set; and adjusting a reserved value of physical storage space of the storage system in response to the determined space consumption meeting a predetermined condition.

13 . The device according to claim 12 , wherein determining space consumption for file system checks of the storage system comprises:
acquiring a second feature set by performing feature extraction on the first feature set based on a feature extraction strategy, wherein the number of features of the second feature set is smaller than that of the first feature set; and determining the space consumption for file system checks of the storage system by the machine learning model based on the second feature set.

14 . The device according to claim 13 , wherein the feature extraction strategy comprises:
merging sizes of logical space of different users in the storage system; merging sizes of physical space of different users in the storage system; and merging namespace object features in the storage system, wherein the namespace object features include volume, snapshot, and clone counts.

15 . The device according to claim 13 , wherein the machine learning model is a pre-trained linear regression model, wherein determining space consumption for file system checks of the storage system comprises:
acquiring a target feature associated with the space consumption for file system checks of the storage system from the second feature set; and inputting the target feature into the linear regression model to determine the space consumption for file system checks of the storage system.

16 . The device according to claim 15 , wherein a method for determining the target feature comprises:
acquiring a first training set for determining the target feature, wherein the first training set comprises at least a size of logical space in the storage system and the space consumption for file system checks of the storage system; determining a degree of correlation between the space consumption for file system checks of the storage system and the remainin

Assignees

• Dell Products Lp

Inventors

• Liu Wenyang
• Gong Chen
• Ni Chunhao
• Tian Ying
• Chi Dapeng

Classifications

• G06F3/0653 Primary

  Monitoring storage devices or systems · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0608 Primary

  Saving storage space on storage systems · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title