Storage apparatus and method for controlling storage apparatus

Patent metadata

Field	Value
Publication number	US-9690487-B2
Application number	US-201514790861-A
Country	US
Kind code	B2
Filing date	Jul 2, 2015
Priority date	Oct 31, 2012
Publication date	Jun 27, 2017
Grant date	Jun 27, 2017

Abstract

Upon receipt of an I/O request instructing storage of data in a storage device 106 from a host apparatus, a storage apparatus 100 selects a de-duplication process method to be applied to the received data, based on at least any of influence on processing performance of the storage apparatus 100 to be performed by execution of a first de-duplication process method (inline method) in which the de-duplication process is performed on the data immediately after the receipt of the I/O request, influence on the processing performance of the storage apparatus 100 to be performed by execution of a second de-duplication process method (post-process method) in which the de-duplication process is performed on the data at later timing, and the size of a temporary storage device 106 b to be required for the processing of the data by the second de-duplication process method.

First claim

The invention claimed is:

1. A storage system comprising:
a storage device configured to provide storage area for storing data sent from an external apparatus,
a memory configured to store information used for an I/O request to the storage device from the external apparatus, and
a processor configured to control the I/O request and to perform a first de-duplication process on data corresponding to the I/O request sent from the external apparatus prior to the data being stored in the storage device and a second de-duplication process on data that has been already stored in a predetermined storage area of the storage device after receiving from the external apparatus,
wherein, upon receipt of the I/O request to target data, the processor is configured to select one of the first or the second de-duplication processes to be applied to the target data based on at least one of information related to a probability of de-duplication of the target data and information related to a capacity of the predetermined storage area in the storage device,
wherein, prior to the second de-duplication process, the processor is configured to divide the target data into a plurality of chunks,
wherein the processor is configured to determine whether any of the plurality of chunks corresponding to the target data matches a universal chunk that is commonly included in a plurality of backup data over multiple backup generations,
wherein, if a result of the determination is negative, the processor is configured to store the specified chunk in the predetermined storage area,
wherein the processor is configured to perform two or more de-duplication processes on two or more respective target data in parallel, in which the two or more de-duplication processes correspond to one of the first and the second de-duplication processes respectively, and

wherein the processor is configured to manage a total number of the two or more de-duplication processes performed in parallel based on at least one of information related to access frequency to the predetermined storage area in the storage device and information related to utilization efficiency of the predetermined storage area in the storage device.

2. The storage system according to claim 1 ,
wherein the processor is configured to provide a file system,
wherein the processor is configured to manage an index of the probability of de-duplication of the target data which is used to select one of the first or the second de-duplication process, on a file basis,
wherein the index is configured to indicate that the first de-duplication process is likely to be selected if at least the part of the file name of the target file matches at least a part of file name of one or more files which have already been stored in the storage device.

3. The storage system according to claim 1 ,
wherein the processor is configured to manage an index of the probability of de-duplication of the target data which is used to select one of the first or the second de-duplication process,
wherein the processor is configured to perform de-duplication process on a part of the target data by comparing the part of the target data and data which has been already stored in the storage device and to determine a de-duplication rate of the part of the target data,
wherein the index is configured to indicate that the first de-duplication process is likely to be selected if the de-duplication rate is higher than a predetermined threshold.

4. The storage system according to claim 1 ,
wherein the processor is configured to manage an index of the probability of de-duplication of the target data which is used to select one of the first or the second de-duplication process,
wherein the index is configured to indicate that the first de-duplication process is likely to be selected, if the target data corresponds to a second or later generation of backup data.

5. The storage system according to claim 1 ,
wherein the processor is configured to manage an index of the probability of de-duplication of the target data which is used to select one of the first or the second de-duplication process,
wherein the index is configured to indicate that the first de-duplication process is likely to be selected if a size of the target data is larger than a predetermined threshold.

6. The storage system according to claim 1 ,
wherein, upon selecting one of the first or the second de-duplication process, the processor is configured to determine whether or not a total capacity of an used capacity of the predetermined storage area and a capacity of the target data exceeds a predetermined threshold,
wherein, if the total capacity of the used capacity of the predetermined storage area and the capacity of the target data exceeds the predetermined threshold, the processor is configured to select the first de-duplication process.

7. The storage system according to claim 1 ,
wherein, subsequently to storing the target data in the predetermined storage area of the storage device, the processor is configured to start the second de-duplication process at a certain time to meet a predetermined requirement of performance and efficiency of the storage apparatus.

8. The storage system according to claim 1 ,
wherein, if the total number of the two or more de-duplication processes performed in parallel exceeds a predetermined threshold, the processor is configured to suspend to start new de-duplication process.

9. The storage system according to claim 1 ,
wherein the processor is configured to start managing the total number of the two or more de-duplication processes performed in parallel, at a time to start or finish one of the de-duplication processes.

10. The storage system according to claim 1 ,
wherein the processor is configured to start managing the total number of the two or more de-duplication processes performed in parallel, in response to receive the target data from the external apparatus.

11. The storage system according to claim 1 ,
wherein the processor is configured to start managing the total number of the two or more de-duplication processes performed in parallel, if the processor detects data in the predetermined storage area of the storage device, on which has not been performed the second de-duplication process.

12. The storage system according to claim 1 ,
wherein, if a load of a network between the storage apparatus and the external apparatus is higher than a predetermined threshold, the processor is configured to increase the number of the second de-duplication processes and to decrease the number of the first de-duplication processes performed in parallel.

13. The storage system according to claim 1 ,
wherein, if a load of a network between the storage apparatus and the external apparatus is higher than a predetermined threshold, the processor is configured to increase the number of the second de-duplication processes and to decrease the number of the first de-duplication processes performed in parallel,
wherein, prior to the second de-duplication process, the processor is configured to compress the target data and to store the compressed data in the predetermined storage area, separately from the second de-duplication process,
wherein, in the second de-duplication process, the processor is configured to uncompress the compressed data stored in the predetermined storage area.

14. The storage system according to claim 1 ,
wherein the processor is configured to increase or decrease at least one of a number of the first de-duplication process and a number of the second de-duplication processes performed in parallel.

Assignees

• Hitachi Ltd
• Hitachi Information & Telecommunication Eng Ltd

Inventors

• Hayasaka Mitsuo
• Yamasaki Koji

Classifications

• G06F3/0641

  De-duplication techniques · CPC title

• G06F3/0619

  in relation to data integrity, e.g. data losses, bit errors · CPC title

• G06F17/30156

  Physics · mapped topic

• G06F3/0608 Primary

  Saving storage space on storage systems · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title