Using prediction uncertainty quantifier with machine learning classifier to predict the survival of a storage device

Patent metadata

Field	Value
Publication number	US-12001968-B2
Application number	US-202117141551-A
Country	US
Kind code	B2
Filing date	Jan 5, 2021
Priority date	Jan 5, 2021
Publication date	Jun 4, 2024
Grant date	Jun 4, 2024

Abstract

The described technology is generally directed towards predicting the survival of a storage device (e.g., a hard disk drive or a solid state drive) to a specified time point, expressed as a confidence score, via a prediction uncertainty quantifier framework with a machine learning classifier. The confidence score corresponds to the likelihood of a storage device surviving until a specified time point (e.g., for n hours). In one implementation, a conformal prediction framework provides the confidence score for a storage device, based on survival rate data predicted using recent telemetry data collected for that storage device by an online semi-parametric Mondrian survival forest classifier. Updated confidence scores based on updated telemetry data can be obtained at various evaluation stages to reevaluate whether to take remedial action with respect to a storage device (e.g., replace the storage device). Multiple storage devices can be ranked by their respective associated confidence scores.

First claim

What is claimed is:

1. A system, comprising:
a processor, and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
selecting a storage device;
obtaining a confidence score corresponding to a predicted survival rate of the storage device for a specified time point, comprising inputting a dataset of parameter data obtained for the storage device into a machine learning model that outputs predicted survival rate data relative to the specified time point, and inputting the predicted survival rate data into a prediction uncertainty quantifier that outputs the confidence score; and
evaluating the confidence score to determine whether to take a remedial action with respect to the storage device prior to the specified time point being reached, wherein the evaluating comprises:
ranking the storage device relative to respective other storage devices based on the confidence score associated with the predicted survival rate of the storage device relative to respective other confidence scores associated with respective other predicted survival rates of the respective other storage devices.

2. The system of claim 1 , wherein the machine learning model comprises a semi-parametric Mondrian survival forest model.

3. The system of claim 2 , wherein the storage device comprises a hard disk drive, and wherein the operations further comprise training the semi-parametric Mondrian survival forest model based on incremental learning using a training dataset obtained from a group of hard disk drives; or
wherein the storage device comprises a solid state drive, and wherein the operations further comprise training the semi-parametric Mondrian survival forest model based on incremental learning using a training dataset obtained from a group of solid state drives.

4. The system of claim 1 , wherein the prediction uncertainty quantifier comprises a conformal prediction framework that obtains a non-conformity measure to determine the confidence score and a credibility score.

5. The system of claim 1 , wherein the prediction uncertainty quantifier comprises a Venn predictor.

6. The system of claim 1 , wherein the machine learning model comprises a semi-parametric Mondrian survival forest model, and wherein the prediction uncertainty quantifier comprises a conformal prediction framework that obtains a non-conformity measure corresponding to the confidence score.

7. The system of claim 1 , wherein the machine learning model comprises a k-nearest neighbor classifier, a random forest classifier, a support vector machines classifier, a neural networks classifier, a logistic regression classifier, or a boosting classifier.

8. The system of claim 1 , wherein the operations further comprise receiving an alert based on simple network management protocol (SNMP) data received from the storage device, and wherein the selecting the storage device is based on the receiving the alert.

9. The system of claim 1 , wherein the specified time point is a first time point associated with a first evaluation stage, wherein the confidence score is a first confidence score, wherein the predicted survival rate is a first predicted survival rate, wherein the dataset of parameter data is a first dataset of parameter data, wherein the predicted survival rate data is a first predicted survival rate data, wherein the remedial action is a first remedial action, and wherein the operations further comprise:
in response to determining not to take the first remedial action with respect to the storage device, in a second evaluation stage, obtaining a second confidence score corresponding to a second predicted survival rate of the storage device for a second specified time point, comprising inputting a second dataset of parameter data obtained for the storage device into the machine learning model that outputs second predicted survival rate data relative to the second specified time point, and inputting the second predicted survival rate data into the prediction uncertainty quantifier that outputs the second confidence score, and evaluating the second confidence score to determine whether to take a second remedial action with respect to the storage device prior to the second specified time point being reached.

10. The system of claim 9 , wherein the operations further comprise passing the first confidence score to the second evaluation stage as a prior confidence score, and wherein evaluating the second confidence score to determine whether to take the second remedial action comprises determining whether the second confidence score relative to the prior confidence score indicates storage device degradation to a specified level.

11. The system of claim 1 , wherein the storage device is a hard disk drive, and wherein the dataset of parameter data comprises at least one of self-monitoring, analysis and reporting technology (SMART) variables, small computer system interface (SCSI) variables, or serial advanced technology attachment (SATA) variables collected with respect to the hard disk drive.

12. The system of claim 1 , wherein the storage device is a solid state drive, and wherein the dataset of parameter data comprises at least one of self-monitoring, analysis and reporting technology (SMART) variables, small computer system interface (SCSI) variables, or serial advanced technology attachment (SATA) variables collected with respect to the solid state drive.

13. A method comprising,
obtaining, at a system comprising a processor, a dataset comprising parameter data of a storage device;
inputting the dataset into a machine learning classifier that outputs prediction data representing a survival rate of the storage device to survive to a specified future time point;
generating, via a conformal prediction framework, a confidence score representing a quality estimate of the prediction data;
associating the confidence score with an identifier of the storage device;
ranking the storage device relative to respective other storage devices based on the confidence score associated with the storage device relative to respective confidence scores associated with the respective other storage devices; and
evaluating the confidence score to determine whether to take a remedial action with respect to the storage device prior to the specified future time point being reached.

14. The method of claim 13 , wherein the inputting the dataset into the machine learning classifier comprises inputting the dataset into an online semi-parametric Mondrian survival forest classifier that produces a forest of decision trees and determines the prediction data based on data associated with individual trees of the forest.

15. The method of claim 13 , further comprising generating, via the conformal prediction framework, a credibility score, and using the credibility score to determine whether the confidence score is credible according to a credibility criterion.

16. The method of claim 13 , wherein the storage device comprises at least one of a hard disk drive or a solid state drive.

17. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
in a first evaluation stage, inputting a first dataset of parameter data obtained for a storage device into an online semi-parametric Mondrian survival forest classifier to obtain first predicted survival rate data of the storage device for a first specified time point;
obtaining a first confidence score representati

Assignees

• Dell Products Lp

Inventors

• Sethi Parminder Singh
• Vishwakarma Rahul Deo
• Sathyanarayanan Vasanth

Classifications

• G06N5/04 Primary

  Inference or reasoning models · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N5/027

  Frames · CPC title

• G06N20/20 Primary

  Ensemble learning · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title