Generation of realistic file content changes for deduplication testing

Patent metadata

Field	Value
Publication number	US-9396203-B2
Application number	US-201314022713-A
Country	US
Kind code	B2
Filing date	Sep 10, 2013
Priority date	Aug 28, 2009
Publication date	Jul 19, 2016
Grant date	Jul 19, 2016

Abstract

Method, system, and computer program product embodiments for facilitating deduplication product testing in a computing environment are provided. In one such embodiment, data to be processed through the deduplication product testing is arranged into a single, continuous stream. At least one of a plurality of random modifications are applied to the arranged data in a self-similar pattern exhibiting scale invariance. A plurality of randomly sized subsets of the arranged data modified with the self-similar pattern is mapped into each of a plurality of randomly sized deduplication test files.

First claim

What is claimed is:

1. A method for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, comprising:
arranging data to be processed through the deduplication product testing into a single, continuous stream; and
applying at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance including:
dividing the arranged data into a plurality of equally sized blocks, and
adding a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

2. The method of claim 1 , wherein the probability distribution includes a Pareto distribution.

3. The method of claim 1 , wherein the dividing the arranged data and the adding the randomly selected variable are performed for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

4. The method of claim 1 , further including providing the plurality of randomly sized deduplication test files to a testing environment in the computing environment, wherein a plurality of randomly sized deduplication test files are accessed by the testing environment.

5. The method of claim 4 , further including calibrating the plurality of randomly sized deduplication test files against input/output (I/O) trace data obtained in the computing environment.

6. A utility for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, comprising:
a deduplication module associated with the processor and memory device in the computing environment, wherein the deduplication module:
arranges data to be processed through the deduplication product testing into a single, continuous stream, and
applies at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance including:
dividing the arranged data into a plurality of equally sized blocks, and
adding a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

7. The utility of claim 6 , wherein the probability distribution includes a Pareto distribution.

8. The utility of claim 6 , wherein the deduplication module performs the dividing the arranged data and the adding the randomly selected variable for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

9. The utility of claim 6 , wherein the plurality of randomly sized deduplication test files are provided to a testing environment in the computing environment, wherein a plurality of randomly sized deduplication test files are accessed by the testing environment.

10. The utility of claim 9 , wherein the deduplication module calibrates the plurality of randomly sized deduplication test files against input/output (I/O) trace data obtained in the computing environment.

11. The utility of claim 6 , wherein the processor comprises a storage management processor responsible for management of a data storage environment associated with the computing environment, and the processor performs a plurality of backup services including one of a full backup operation and an incremental backup operation.

12. A computer program product for facilitating deduplication product testing in a computing environment by a processor in communication with a memory device, the computer program product comprising a computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:
a first executable portion that arranges data to be processed through the deduplication product testing into a single, continuous stream; and
a second executable portion that applies at least one of a plurality of random modifications to the arranged data in a self-similar pattern exhibiting scale invariance including:
dividing the arranged data into a plurality of equally sized blocks, and
adding a randomly selected variable obeying a probability distribution to one of the plurality of equally sized blocks.

13. The computer program product of claim 12 , wherein the probability distribution includes a Pareto distribution.

14. The computer program product of claim 12 , wherein the dividing the arranged data and the adding the randomly selected variable are performed for each of the plurality of equally sized blocks while a number of the plurality of equally sized blocks is less than a desired stream length.

15. The computer program product of claim 12 , further including a third executable portion that provides the plurality of randomly sized deduplication test files to a testing environment in the computing environment, wherein a plurality of randomly sized deduplication test files are accessed by the testing environment.

16. The computer program product of claim 15 , further including a fourth executable portion that calibrates the plurality of randomly sized deduplication test files against input/output (I/O) trace data obtained in the computing environment.

Assignees

• IBM

Inventors

• Mcnutt Bruce

Classifications

• G06F16/1748 Primary

  De-duplication implemented within the file system, e.g. based on file segments (de-duplication techniques in storage systems for the management of data blocks G06F3/0641) · CPC title

• G06F16/13

  File access structures, e.g. distributed indices (arrangements of input from, or output to, record carriers G06F3/06) · CPC title

• G06F17/30156 Primary

  Physics · mapped topic