Systems and methods for data storage remediation

Patent metadata

Field	Value
Publication number	US-9354971-B2
Application number	US-201414260109-A
Country	US
Kind code	B2
Filing date	Apr 23, 2014
Priority date	Apr 23, 2014
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

Systems, methods, and non-transitory computer-readable media can receive an indication that a drive included in a set of drives has experienced an error. Acquiring first data associated with the drive can be attempted. The drive can be prompted to undergo a power-cycle. Acquiring second data associated with the drive can be attempted. At least one of the first data or the second data can be analyzed to determine that the drive is remediable. At least the drive included in the set of drives can be remediated, such as by being rebuilt or reconfigured.

First claim

What is claimed:

1. A computer-implemented method comprising:
receiving, by a computing system, an indication that a drive included in a set of drives has experienced an error;
attempting, by the computing system, to acquire first data associated with the drive;
prompting, by the computing system, the drive to undergo a power-cycle;
attempting, by the computing system, to acquire second data associated with the drive, wherein attempting to acquire the second data occurs subsequent to the power-cycle, and wherein attempting to acquire the first data occurs prior to the power-cycle;
analyzing, by the computing system, at least one of the first data or the second data to determine that the drive is remediable; and
remediating, by the computing system, at least the drive included in the set of drives.

2. The computer-implemented method of claim 1 , wherein the remediating at least the drive included in the set of drives further comprises:
attempting to acquire third data associated with the drive;
analyzing the third data to verify that the drive is operational; and
performing at least one of a rebuild or a reconfiguration with respect to one or more file systems associated with at least the drive included in the set of drives.

3. The computer-implemented method of claim 1 , wherein the analyzing the at least one of the first data or the second data to determine that the drive is remediable further comprises comparing the first data and the second data to determine one or more differences, the one or more differences being indicative that the drive has at least a threshold likelihood of being remediable.

4. The computer-implemented method of claim 1 , wherein each of the first data and the second data includes Self-Monitoring, Analysis and Reporting Technology (SMART) data.

5. The computer-implemented method of claim 4 , wherein the SMART data includes information associated with at least one of a grown defects list (GLIST), an error-correcting code (ECC) memory, or a quantity of uncorrectable errors.

6. The computer-implemented method of claim 1 , wherein the set of drives corresponds to a Redundant Array of Independent Disks (RAID) system.

7. The computer-implemented method of claim 1 , further comprising:
removing at least a portion of historical data associated with the drive prior to the remediating at least the drive included in the set of drives.

8. The computer-implemented method of claim 7 , wherein the removing at least the portion of the historical data associated with the drive comprises at least one of clearing a cache associated with the drive, clearing configuration data associated with the drive, or removing metadata associated with the drive.

9. The computer-implemented method of claim 8 , wherein the set of drives is associated with a RAID 6 system.

10. The computer-implemented method of claim 1 , further comprising:
performing a scan on the drive to verify that the drive is remediable, wherein the scan is performed prior to the remediating at least the drive included in the set of drives.

11. The computer-implemented method of claim 10 , wherein the set of drives is associated with a RAID 0 system.

12. The computer-implemented method of claim 10 , wherein the scan is associated with a bad block scan.

13. The computer-implemented method of claim 12 , wherein the bad block scan is associated with a destructive scan.

14. The computer-implemented method of claim 1 , wherein the analyzing the at least one of the first data or the second data to determine that the drive is remediable further comprises:
determining an identifier associated with the drive;
acquiring information associated with the identifier; and
determining that the drive had not previously experienced the error.

15. The computer-implemented method of claim 1 , further comprising:
recording, over time, at least one of the first data or the second data;
analyzing the at least one of the first data or the second data to determine a first set of properties associated with a first category of drives; and
categorizing another drive having the first set of properties as being irremediable.

16. The computer-implemented method of claim 1 , further comprising:
recording, over time, at least one of the first data or the second data;
analyzing the at least one of the first data or the second data to determine a second set of properties associated with a second category of drives; and
categorizing another drive having the second set of properties as being remediable.

17. The computer-implemented method of claim 1 , wherein the error is associated with at least one of a firmware fail, an Input/Output (IO) error, or a file system error.

18. The computer-implemented method of claim 1 , wherein the indication is received via at least one of a RAID controller, a system health checker, or a monitoring component associated with a social networking service.

19. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
receiving an indication that a drive included in a set of drives has experienced an error;
attempting to acquire first data associated with the drive;
prompting the drive to undergo a power-cycle;
attempting to acquire second data associated with the drive, wherein attempting to acquire the second data occurs subsequent to the power-cycle, and wherein attempting to acquire the first data occurs prior to the power-cycle;
analyzing at least one of the first data or the second data to determine that the drive is remediable; and
remediating at least the drive included in the set of drives.

20. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:
receiving an indication that a drive included in a set of drives has experienced an error;
attempting to acquire first data associated with the drive;
prompting the drive to undergo a power-cycle;
attempting to acquire second data associated with the drive, wherein attempting to acquire the second data occurs subsequent to the power-cycle, and wherein attempting to acquire the first data occurs prior to the power-cycle;
analyzing at least one of the first data or the second data to determine that the drive is remediable; and
remediating at least the drive included in the set of drives.

Assignees

• Facebook Inc

Inventors

• Brauer Jonathan Charles
• Niewczas Mateusz Marek
• Tal Eran

Classifications

• G06F11/0793 Primary

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• G06F11/1076 Primary

  Parity data used in redundant arrays of independent storages, e.g. in RAID systems · CPC title

• G06F11/3034

  where the computing system component is a storage system, e.g. DASD based or network based (digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097) · CPC title

• G06F11/1084

  Degraded mode, e.g. caused by single or multiple storage removals or disk failures · CPC title

• G06F11/006

  Identification (G06F11/2289 takes precedence) · CPC title