Reliable and scalable image transfer for data centers with low connectivity using redundancy detection

Patent metadata

Field	Value
Publication number	US-9274818-B2
Application number	US-201313760125-A
Country	US
Kind code	B2
Filing date	Feb 6, 2013
Priority date	Feb 6, 2013
Publication date	Mar 1, 2016
Grant date	Mar 1, 2016

Abstract

A system and method for efficiently transferring virtual machine images across nodes in a cloud computing environment, includes analyzing each image on each node to create hash code clusters and a similarity matrix. An instruction to transfer an image from a source node to a target node is received. The clusters and the similarity matrix are used to determine to what extent the data from the image is already on the source node, or on any other node, and further determines the cost and speed of transferring such data to the target node. An optimal transfer plan is generated, and data that is not already on the target node is transferred to the target node from the most efficient node on which it is available, according to the optimal transfer plan.

First claim

What is claimed is:

1. A method for transferring virtual machine images between a plurality of nodes, comprising:
analyzing each of a plurality of virtual machine images present on a plurality of nodes to create image clusters corresponding to a plurality of data blocks present in the plurality of virtual machine images, using a program executable by a processor communicating with a computer, the plurality of nodes including a source node and a target node, wherein each image cluster is associated with a grouping of data blocks and comprises a list of hash codes of the data blocks and further comprises an image cluster identifier;
generating a similarity matrix using the created image clusters;
selecting a virtual machine image present on the source node to be reconstituted on the target node;
identifying a plurality of data block sets in the selected virtual machine image for transfer to the target node, using the similarity matrix;
transferring the plurality of data block sets to the target node, according to an optimal transfer plan; and
reconstituting the plurality of data block sets on the target node using at least the transferred plurality of data block sets,
wherein the image cluster identifier comprises a bit string having a length at least equal to a count of the plurality of virtual machine images, wherein each bit corresponds to one of the plurality of virtual machine images, and each bit having a value of (1) indicates that the image cluster is present on its corresponding virtual machine image.

2. The method of claim 1 , wherein the data block sets are distributed on a plurality of nodes.

3. The method of claim 1 , wherein the data block sets are on both the source node and the target node.

4. The method of claim 1 , wherein the data block sets are only on the source node.

5. The method of claim 1 , further comprising:
identifying a first data block set from the selected virtual machine image that are present only on the source node;
identifying a second data block set from the selected virtual machine image that are present on the source node, and on at least another node excluding the target node;
identifying a third data block set from the selected virtual machine image that are present on the source node, and on at least the target node;
transferring the first and second data block sets from the source node or at least another node, excluding the target node, to the target node, according to an optimal plan; and
reconstituting on the target node the selected virtual machine image, using the transferred first and second data block sets, and the third data block set present on the target node.

6. The method of claim 1 , wherein the plurality of nodes are datacenters within a public or private cloud computing environment.

7. The method of claim 1 , wherein the number of image clusters used to generate a similarity matrix is configurable.

8. The method of claim 1 , wherein the optimal transfer plan is determined by minimizing a time factor, a bandwidth factor, and a cost factor, using the similarity matrix.

9. The method of claim 1 , wherein the plurality of nodes comprises a group of systems categorized by cost and/or bandwidth requirements.

10. A system for transferring virtual machine images, comprising:
one or more computer having a processor processors;
one or more non-transitory computer-readable storage device media;
program instructions embodied on the storage device stored on the non-transitory computer-readable storage media for execution by at least one of the one or more processor processors, the program instructions comprising:
program instructions to analyze each of a plurality of virtual machine images present on a plurality of nodes to create image clusters corresponding to a plurality of data blocks present in the plurality of virtual machine images, the plurality of nodes including a source node and a target node, wherein each image cluster is associated with a grouping of data blocks and comprises a list of hash codes of the data blocks and further comprises an image cluster identifier;
program instructions to generate a similarity matrix using the created image clusters;
program instructions to select a virtual machine image present on the source node to be reconstituted on the target node;
program instructions to identify a plurality of data block sets in the selected virtual machine image for transfer to the target node, using the similarity matrix;
program instructions to transfer the plurality of data block sets to the target node, according to an optimal transfer plan; and
program instructions to reconstitute the plurality of data block sets on the target node, using at least the transferred plurality of data block sets,

wherein the image cluster identifier comprises a bit string having a length at least equal to a count of the plurality of virtual machine images, wherein each bit corresponds to one of the plurality of virtual machine images, and each bit having a value of (1) indicates that the image cluster is present on its corresponding virtual machine image.

11. The system of claim 10 , wherein the data block sets are distributed on a plurality of nodes.

12. The system of claim 10 , wherein the data block sets are on both the source node and the target node.

13. The system of claim 10 , wherein the data block sets are only on the source node.

14. The system of claim 10 , wherein the plurality of nodes are datacenters within a public or private cloud computing environment.

15. A computer program product for transferring virtual machine images, the computer program product comprising:
a non-transitory computer-readable storage medium having program code embodied therewith, the program code readable/executable by a processor to perform a method comprising:
analyzing each of a plurality of virtual machine images present on a plurality of nodes, by the processor, to create image clusters corresponding to a plurality of data blocks present in the plurality of virtual machine images, the plurality of nodes including a source node and a target node, wherein each image cluster is associated with a grouping of data blocks and comprises a list of hash codes of the data blocks and further comprises an image cluster identifier;
generating a similarity matrix, by the processor, using the created image clusters;
selecting a virtual machine image present on the source node, by the processor, to be reconstituted on the target node;
identifying a plurality of data block sets in the selected virtual machine image, by the processor, for transfer to the target node, using the similarity matrix;
transferring the plurality of data block sets to the target node, by the processor, according to an optimal transfer plan; and
reconstituting the plurality of data block sets, by the processor, on the target node using at least the transferred plurality of data block sets,
wherein the image cluster identifier comprises a bit string having a length at least equal to a count of the plurality of virtual machine images, wherein each bit corresponds to one of the plurality of virtual machine images, and each bit having a value of (1) indicates that the image cluster is present on its corresponding virtual machine image.

16. The computer program product of claim 15 , wherein the data blocks sets are distributed on a plurality of nodes.

17. The computer program product of claim 15 , wherein the data block sets are on both the source node and the target node.

18. The computer program product of claim 15 , wherein the

Assignees

• IBM

Inventors

• Karve Alexei A
• Kochut Andrzej
• Shaikh Hidayatullah
• Hrischuk Curtis E

Classifications

• G06F9/455 Primary

  Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines · CPC title

• G06F9/461 Primary

  Saving or restoring of program or task context · CPC title

• G06F9/4856

  resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence) · CPC title

• G06F11/1484

  involving virtual machines · CPC title

• G06F9/45558 Primary

  Hypervisor-specific management and integration aspects · CPC title