Systems and methods for supporting migration of virtual machines accessing remote storage devices over network via NVMe controllers

Patent metadata

Field	Value
Publication number	US-9430268-B2
Application number	US-201414537758-A
Country	US
Kind code	B2
Filing date	Nov 10, 2014
Priority date	May 2, 2014
Publication date	Aug 30, 2016
Grant date	Aug 30, 2016

Abstract

A new approach is proposed virtual machines (VMs) accessing remote storage devices over a network via non-volatile memory express (NVMe) controllers to migrate live from a current host to a destination host. A first virtual NVMe controller running on a first physical NVMe controller enables a first VM running on the current host to perform storage operations to logical volumes mapped to the remote storage devices over the network as if they were local storage volumes. During VM migration, the current host puts the first virtual NVMe controller into quiesce state and saves an image of its states. A second virtual NVMe controller is created on a second physical NVMe controller using the saved image, which is configured to serve a second VM on the destination host. The second virtual NVMe controller resumes the storage operations to the remote storage devices without being interrupted by the VM migration.

First claim

What is claimed is:

1. A computer-implemented method to support migration of virtual machines (VMs) accessing a set of remote storage devices over a network via non-volatile memory express (NVMe) controllers, comprising:
enabling a first VM running on a first host to access and perform a plurality of storage operations to one or more logical volumes in one or more NVMe namespaces created and mapped to the remote storage devices accessible by a first virtual NVMe controller only over the network as if they were local storage volumes following a storage network protocol;
putting said first virtual NVMe controller running on a first physical NVMe controller currently serving the first VM into a quiesce state when the first VM is being migrated from the first host to a second VM running on a second host;
capturing and saving an image of states of the first virtual NVMe controller on the first host;
creating a second virtual NVMe controller on a second physical NVMe controller using the saved image, wherein the second virtual NVMe controller is configured to serve the second VM and has exactly the same states as the first virtual NVMe controller in the quiesce state;
initiating and/or resuming the storage operations to the logical volumes mapped to the remote storage devices accessible by the second virtual NVMe controller over the network without being interrupted by the migration of the first VM running on the first host to the second VM running on the second host.

2. The method of claim 1 , wherein:
the first host is coupled to the first physical NVMe controller and the second host is coupled to the second physical NVMe controller.

3. The method of claim 1 , wherein:
the first physical NVMe controller and the second physical NVMe controller are different.

4. The method of claim 1 , wherein:
the logical volumes further include storage devices attached to the physical NVMe controller locally.

5. The method of claim 1 , wherein:
the storage network protocol is Internet Small Computer System Interface (iSCSI).

6. The method of claim 1 , wherein:
the states of the first virtual NVMe controller include status of the pending storage operations initiated by the first VM to the logical volumes mapped to the remote storage devices and/or network connections to the remote storage devices.

7. The method of claim 1 , wherein:
the first virtual NVMe controller is temporarily inactive or disabled and stops to serve the storage operations to the remote storage devices at the quiesce state.

8. The method of claim 1 , further comprising:
utilizing and programming a set of registers per the first virtual NVMe controller on the first physical NVMe controller to maintain the states information of the first virtual NVMe controller during its normal operation.

9. The method of claim 1 , further comprising:
allowing the first virtual NVMe controller running on the first physical NVMe controller to be migrated on a per Virtual Function (VF)-basis, where only the states of the first virtual NVMe controller currently serving the first VM is migrated/copied to the second virtual NVMe controller on the second physical NVMe controller while other virtual NVMe controller(s) running on the first physical NVMe controller are not affected and/or migrated.

10. The method of claim 1 , further comprising:
resetting the states of the first virtual NVMe controller and re-assigning it to serve another VM after the first VM has been migrated from the first host to the second host and the second virtual NVMe controller has been instantiated to serve the migrated second VM.

11. The method of claim 1 , further comprising:
reading from the image and programming a set of registers per the second virtual NVMe controller on the first physical NVMe controller to bring the second virtual NVMe controller to the states that existed when the first virtual NVMe controller was in the quiesce state.

12. The method of claim 1 , further comprising:
mapping and exporting the same set of remote storage devices to the logical volumes to the first and the second VMs via the first and the second virtual NVMe controllers, respectively, using the storage network protocol.

13. A system to support migration of virtual machines (VMs) accessing a set of remote storage devices over a network via non-volatile memory express (NVMe) controllers, comprising:
a first physical NVMe controller including a first virtual NVMe controller running on it, which in operation, is configured to:
enable a first VM running on a first host to access and perform a plurality of storage operations to one or more logical volumes in one or more NVMe namespaces created and mapped to the remote storage devices accessible by the first virtual NVMe controller only over the network as if they were local storage volumes following a storage network protocol;

said first host, which in operation, is configured to:
put the first virtual NVMe controller currently serving the first VM into a quiesce state when the first VM is being migrated from the first host to a second VM running on a second host;
capture and save an image of states of the first virtual NVMe controller on the first host;
create a second virtual NVMe controller on a second physical NVMe controller using the saved image, wherein the second virtual NVMe controller is configured to serve the second VM and has exactly the same states as the first virtual NVMe controller in the quiesce state;

said second physical NVMe controller including said second virtual NVMe controller running on it, which in operation, is configured to:
initiate and/or resume the storage operations to the logical volumes mapped to the remote storage devices accessible by the second virtual NVMe controller over the network without being interrupted by the migration of the first VM running on the first host to the second VM running on the second host.

14. The system of claim 13 , wherein:
each of the first and the host is an x86/ARM server.

15. The system of claim 13 , wherein:
the first host is coupled to the first physical NVMe controller and the second host is coupled to the second physical NVMe controller.

16. The system of claim 13 , wherein:
the first physical NVMe controller and the second physical NVMe controller are different.

17. The system of claim 13 , wherein:
the logical volumes further include storage devices attached to the physical NVMe controller locally.

18. The system of claim 13 , wherein:
the storage network protocol is Internet Small Computer System Interface (iSCSI).

19. The system of claim 13 , wherein:
the states of the first virtual NVMe controller include status of the pending storage operations initiated by the first VM to the logical volumes mapped to the remote storage devices and/or network connections to the remote storage devices.

20. The system of claim 13 , wherein:
the first virtual NVMe controller is temporarily inactive or disabled and stops to serve the storage operations to the remote storage devices at the quiesce state.

21. The system of claim 13 , wherein:
the first virtual NVMe controller is configured to utilize and program a set of registers per the first virtual NVMe controller on the first physical NVMe controller to maintain the states information of the first virtual NVMe controller during its normal operation.

22. The system of claim 13 , wherein:
the first host is configured to allow the first virtual NVMe controller running on the fir

Assignees

• Cavium Inc

Inventors

• Hussain Muhammad Raghib
• Murgai Vishal
• Panicker Manojkumar
• Masood Faisal
• Folsom Brian
• Kessler Richard Eugene

Classifications

• H04L67/1097

  for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F2009/4557

  Distribution of virtual machine instances; Migration and load balancing · CPC title

• G06F9/45537

  Provision of facilities of other operating environments, e.g. WINE (I/O emulation G06F13/105) · CPC title