Compression-based detection of inefficiency in local storage

Patent metadata

Field	Value
Publication number	US-2017336984-A1
Application number	US-201615160898-A
Country	US
Kind code	A1
Filing date	May 20, 2016
Priority date	May 20, 2016
Publication date	Nov 23, 2017
Grant date	—

Abstract

The disclosed embodiments provide a system for detecting and managing inefficiency in local storage. During operation, the system obtains a first snapshot of data in local storage of a computer system, wherein the first snapshot comprises a first set of data elements in the local storage at a first time. Next, the system applies a compression technique to the first snapshot to obtain a first set of inefficiency metrics for the first set of data elements. The system then outputs the first set of inefficiency metrics with additional attributes of the data to improve management of inefficiency in the data.

First claim

What is claimed is:

1 . A method, comprising:
obtaining a first snapshot of data in local storage of a computer system, wherein the first snapshot comprises a first set of data elements in the local storage at a first time; applying, by a processor, a compression technique to the first snapshot to obtain a first set of inefficiency metrics for the first set of data elements; and outputting the first set of inefficiency metrics with additional attributes of the data to improve management of inefficiency in the data.

2 . The method of claim 1 , further comprising:
grouping a subset of the first set of data elements by an attribute; applying the compression technique to the grouped subset to obtain a group inefficiency metric for the grouped subset; and including the group inefficiency metric in the outputted set of inefficiency metrics.

3 . The method of claim 2 , wherein the attribute comprises at least one of:
a file name; a file type; a data type; a directory; a device; a service; and an executable.

4 . The method of claim 1 , further comprising:
obtaining a difference between the first snapshot and a second snapshot of the data in the local storage, wherein the second snapshot comprises a second set of data elements in the local storage at a second time; applying the compression technique to the difference to obtain a second set of inefficiency metrics for the difference; and analyzing the first and second sets of inefficiency metrics to identify a type of inefficiency in the data.

5 . The method of claim 4 , further comprising:
adjusting an interval between the first and second snapshots based on the first or second sets of inefficiency metrics.

6 . The method of claim 4 , wherein the type of inefficiency is at least one of:
data fragmentation; a logging inefficiency; an input/output (I/O) inefficiency; and a schema inefficiency.

7 . The method of claim 1 , wherein the first set of data elements comprises at least one of:
a file; a log; a record; a write; and a read.

8 . The method of claim 1 , wherein the local storage is at least one of:
a hard disk drive (HDD); a solid-state drive; an optical drive; and a tape drive.

9 . The method of claim 1 , wherein the set of inefficiency metrics comprises at least one of:
a redundancy; and a compression ratio.

10 . The method of claim 1 , wherein outputting the set of inefficiency metrics with the additional attributes of the first set of data elements comprises at least one of:
displaying a ranking of the first set of data elements by the first set of inefficiency metrics; and identifying a subset of the first set of data elements as candidates for improving the inefficiency.

11 . An apparatus, comprising:
one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a first snapshot of data in local storage of a computer system, wherein the first snapshot comprises a first set of data elements in the local storage at a first time;
apply a compression technique to the first snapshot to obtain a first set of inefficiency metrics for the first set of data elements; and
output the first set of inefficiency metrics with additional attributes of the data to improve management of inefficiency in the data.

12 . The apparatus of claim 11 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
group a subset of the first set of data elements by an attribute; apply the compression technique to the grouped subset to obtain a group inefficiency metric for the grouped subset; and include the group inefficiency metric in the outputted set of inefficiency metrics.

13 . The apparatus of claim 12 , wherein the attribute comprises at least one of:
a file name; a file type; a data type; a directory; a device; a service; and an executable.

14 . The apparatus of claim 11 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
obtain a difference between the first snapshot and a second snapshot of the data in the local storage, wherein the second snapshot comprises a second set of data elements in the local storage at a second time; apply the compression technique to the difference to obtain a second set of inefficiency metrics for the difference; and analyze the first and second sets of inefficiency metrics to identify a type of inefficiency in the data.

15 . The apparatus of claim 14 , wherein the memory further stores instructions that, when executed by the one or more processors, cause the apparatus to:
adjust an interval between the first and second snapshots based on the first or second sets of inefficiency metrics.

16 . The apparatus of claim 14 , wherein the type of inefficiency is at least one of:
data fragmentation; a logging inefficiency; an input/output (I/O) inefficiency; and a schema inefficiency.

17 . The apparatus of claim 14 , wherein the first set of data elements comprises at least one of:
a file; a log; a record; a write; and a read.

18 . A system, comprising:
an analysis module comprising a non-transitory computer-readable medium storing instructions that, when executed by, cause the system to:
obtain a first snapshot of data in local storage of a computer system, wherein the first snapshot comprises a first set of data elements in the local storage at a first time; and
apply a compression technique to the first snapshot to obtain a first set of inefficiency metrics for the first set of data elements; and
a management module comprising a non-transitory computer-readable medium storing instructions that, when executed, cause the system to output the first set of inefficiency metrics with additional attributes of the data to improve management of inefficiency in the data.

19 . The system of claim 18 , wherein the non-transitory computer- readable medium of the analysis module further stores instructions that, when executed, cause the system to:
obtain a difference between the first snapshot and a second snapshot of the data in the local storage, wherein the second snapshot comprises a second set of data elements in the local storage at a second time; apply the compression technique to the difference to obtain a second set of inefficiency metrics for the difference; and analyze the first and second sets of inefficiency metrics to identify a type of inefficiency in the data.

20 . The system of claim 19 , wherein the non-transitory computer- readable medium of the analysis module further stores instructions that, when executed, cause the system to:
adjust an interval between the first and second snapshots based on the first or second sets of inefficiency metrics.

Assignees

• Linkedin Corp

Inventors

• Nicol John W
• Maheshwari Ritesh
• Baggott Nicholas P
• Ramachandra Haricharan K
• Sridharan Badrinath K

Classifications

• G06F3/062

  Securing storage systems · CPC title

• G06F3/0673

  Single storage device · CPC title

• G06F3/0608 Primary

  Saving storage space on storage systems · CPC title

• G06F3/0661

  Format or protocol conversion arrangements · CPC title

• G06F3/0643

  Management of files · CPC title