Generation of realistic file content changes for deduplication testing

Patent metadata

Field	Value
Publication number	US-9633034-B2
Application number	US-201615183026-A
Country	US
Kind code	B2
Filing date	Jun 15, 2016
Priority date	Aug 28, 2009
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

Method, system, and computer program product embodiments for facilitating deduplication product testing in a computing environment are provided. In one such embodiment, data to be processed through the deduplication product testing is arranged into a single, continuous stream. At least one of a plurality of random modifications are applied to the arranged data in a self-similar pattern exhibiting scale invariance. A plurality of randomly sized subsets of the arranged data modified with the self-similar pattern is mapped into each of a plurality of randomly sized deduplication test files which are calibrated against input/output (I/O) trace data obtained in the computing environment.

First claim

What is claimed is:

1. A method for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, comprising:
arranging data to be processed through the deduplication product testing into a single, continuous stream;
applying at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance;
mapping a plurality of randomly sized subsets of the arranged data modified with the self-similar pattern into each of a plurality of randomly sized deduplication test files; and
calibrating the plurality of randomly sized deduplication test files against input/output (PO) trace data of physical I/O references in obtained in the computing environment in lieu of requiring a file system layout be provided to perform the calibration.

2. The method of claim 1 , wherein the applying the at least one of the plurality of random modifications to the arranged data in the self-similar pattern includes:
dividing the arranged data into a plurality of equally sized blocks, and
adding a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

3. The method of claim 2 , wherein the probability distribution includes a Pareto distribution.

4. The method of claim 2 , wherein the dividing the arranged data and the adding the randomly selected variable are performed for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

5. The method of claim 1 , further including providing the plurality of randomly sized deduplication test files to a testing environment in the computing environment, wherein the plurality of randomly sized deduplication test files are accessed by the testing environment.

6. A utility for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, comprising:
a deduplication module associated with the processor and memory device in the computing environment, wherein the deduplication module:
arranges data to be processed through the deduplication product testing into a single, continuous stream,
applies at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance,
maps a plurality of randomly sized subsets of the arranged data modified with the self-similar pattern into each of a plurality of randomly sized deduplication test files, and
calibrates the plurality of randomly sized deduplication test files against input/output (I/O) trace data of physical I/O references in obtained in the computing environment in lieu of requiring a file system layout be provided to perform the calibration.

7. The utility of claim 6 , wherein the deduplication module, pursuant to the applying the at least one of the plurality of random modifications to the arranged data in the self-similar pattern:
divides the arranged data into a plurality of equally sized blocks, and
adds a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

8. The utility of claim 7 , wherein the probability distribution includes a Pareto distribution.

9. The utility of claim 7 , wherein the deduplication module performs the dividing the arranged data and the adding the randomly selected variable for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

10. The utility of claim 6 , wherein the plurality of randomly sized deduplication test files are provided to a testing environment in the computing environment, wherein the plurality of randomly sized deduplication test files are accessed by the testing environment.

11. The utility of claim 6 , wherein the processor comprises a storage management processor responsible for management of a data storage environment associated with the computing environment, and the processor performs a plurality of backup services including one of a full backup operation and an incremental backup operation.

12. A computer program product for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
an executable portion that arranges data to be processed through the deduplication product testing into a single, continuous stream;
an executable portion that applies at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance;
an executable portion that maps a plurality of randomly sized subsets of the arranged data modified with the self-similar pattern into each of a plurality of randomly sized deduplication test files; and
an executable portion that calibrates the plurality of randomly sized deduplication test files against input/output (I/O) trace data of physical I/O references in obtained in the computing environment in lieu of requiring a file system layout be provided to perform the calibration.

13. The computer program product of claim 12 , wherein the applying the at least one of the plurality of random modifications to the arranged data in the self-similar pattern includes an executable portion that:
divides the arranged data into a plurality of equally sized blocks, and
adds a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

14. The computer program product of claim 13 , wherein the probability distribution includes a Pareto distribution.

15. The computer program product of claim 13 , wherein the dividing the arranged data and the adding the randomly selected variable are performed for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

16. The computer program product of claim 12 , further including an executable portion that provides the plurality of randomly sized deduplication test files to a testing environment in the computing environment, wherein the plurality of randomly sized deduplication test files are accessed by the testing environment.

Assignees

• IBM

Inventors

• Mcnutt Bruce

Classifications

• G06F16/1748 Primary

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F16/13

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title

• G06F17/30156 Primary

  Physics · mapped topic

• G06F17/30091

  Physics · mapped topic