Remote-direct-memory-access-based virtual machine live migration

Patent metadata

Field	Value
Publication number	US-2016378530-A1
Application number	US-201514835762-A
Country	US
Kind code	A1
Filing date	Aug 26, 2015
Priority date	Jun 27, 2015
Publication date	Dec 29, 2016
Grant date	—

Abstract

The current document is directed to methods and systems for moving executing virtual machines between host systems in a virtual data center. In described implementations, remote-direct memory access is used for transferring memory contents and, in certain implementations, additional data between the host systems to facilitate live migration of virtual machines. To provide increased efficiency, transfer of the contents of a shared memory page from a source host system to target host system during migration of a virtual machine is deferred until the relocated virtual machine attempts to write to the shared memory page.

First claim

1 . A virtualized distributed computing system comprising:
a first host system that includes one or more processors, one or more memories, and a management-server agent, the first host system executing an original virtual machine and interconnected with the management server through a network; a second host system that includes one or more processors, one or more memories, and a management-server agent, the second host system interconnected with the first host system and the management server through the network; and the management server, which coordinates a live-virtual-machine migration operation that moves the original virtual machine to the second host system, during which the contents of one or more memory pages are transferred from physical memory within the first host system to a physical memory within the second host system through a remote-data memory access interface to a network-interface controller on one or both of the first and second host systems.

2 . The virtualized distributed computing system of claim 1 wherein the original virtual machine is moved from the first host system to become a relocated virtual machine executing on the second host system with minimal interruption in execution, on the order of less than one to a few seconds.

3 . The virtualized distributed computing system of claim 1 wherein the original virtual machine is moved from the first host system to become a relocated virtual machine executing on the second host system by:
moving an execution image of the virtual machine from the first host system to the second host system;
moving swap files of the virtual machine from the first host system to the second host system;
moving shared-memory-page and non-shared-memory-page lists for the virtual machine from the first host system to the second host system;
moving modified physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system;
terminating execution of the original virtual machine on the first host system; and
resuming execution of the relocated virtual machine on the second host computer.

4 . The virtualized distributed computing system of claim 3 wherein moving modified physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system further comprises moving, through the remote-data memory access interface to the network-interface controller on one or both of the first and second host systems, modified non-shared physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system.

5 . The virtualized distributed computing system of claim 4 wherein each shared memory page is described by a shared-memory-page data structure.

6 . The virtualized distributed computing system of claim 5 wherein each shared-memory-page data structure includes:
a local reference count that indicates the number of virtual machines within the host computer system in which the shared-memory-page data structure is stored;
a reference to a list of virtual-page-number data structures, each virtual-page-number data structure including indications of the virtual machine and the virtual page number for the shared memory page described by the shared-memory-page data structure in the address space of the guest operating system of the virtual machine;
a remote reference count that indicates the number of virtual machines within host systems remote to the host computer system in which the shared-memory-page data structure is stored; and
a reference to a list of virtual-page-number data structures, each virtual-page-number data structure including indications of the virtual machine and the virtual page number for the shared memory page described by the shared-memory-page data structure in the address space of the guest operating system of the virtual machine.

7 . The virtualized distributed computing system of claim 4 wherein shared physical-memory pages to which guest physical pages of the virtual machine are mapped remain mapped to the physical memory of the first host system during and after the original virtual machine is move to the second host system and execution of the relocated virtual machine is resumed.

8 . The virtualized distributed computing system of claim 7 wherein, when the relocated virtual machine accesses a shared memory page that is mapped to the first host system, following resumption of execution of the relocated virtual machine, the contents of the shared memory are obtained by the virtualization layer second host system and placed in the physical memory of the second host system through the remote-data memory access interface to the network-interface controller on the second host system.

9 . The virtualized distributed computing system of claim 8 wherein, when the relocated virtual machine attempts to write a value to a shared memory page that is mapped to the first host system, following resumption of execution of the relocated virtual machine, the shared memory page is changed to be a non-shared memory page on the second host system and a remote reference count within a shared-memory-page data structure within the first host system is decremented.

10 . A method that effects a live-virtual-machine migration within a virtualized distributed computing system, the virtualized distributed computing system including a first host system that includes one or more processors, one or more memories, and a management-server agent, the first host system executing an original virtual machine and interconnected with the management server through a network, a second host system that includes one or more processors, one or more memories, and a management-server agent, the second host system interconnected with the first host system and the management server through the network, and a management server, which coordinates a live-virtual-machine migration operation that moves the original virtual machine to the second host system, the method comprising:
moving an execution image of the virtual machine from the first host system to the second host system; moving swap files of the virtual machine from the first host system to the second host system; moving shared-memory-page and non-shared-memory-page lists for the virtual machine from the first host system to the second host system; moving modified physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system; terminating execution of the original virtual machine on the first host system; and resuming execution of the relocated virtual machine on the second host computer.

11 . The method of claim 10 wherein the original virtual machine is moved from the first host system to become a relocated virtual machine executing on the second host system with minimal interruption in execution, on the order of less than one to a few seconds.

12 . The method of claim 10 wherein moving modified physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system further comprises moving, through the remote-data memory access interface to the network-interface controller on one or both of the first and second host systems, modified non-shared physical-memory pages to which guest physical pages of the virtual machine are mapped from the first host system to the second host system.

13 . The method of claim 12 wherein each shared memory page is described by a shared-memory-page data structure.

14 . The

Assignees

• Vmware Inc

Inventors

• Ramasubramanian Manikandan
• Ahmed Mukheem

Classifications

• G06F2009/45595

  Network integration; Enabling network access in virtual machine instances · CPC title

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• G06F9/45558 Primary

  Hypervisor-specific management and integration aspects · CPC title

• G06F2009/45583

  Memory management, e.g. access or allocation · CPC title