Systems and methods for NVMe controller virtualization to support multiple virtual machines running on a host

Patent metadata

Field	Value
Publication number	US-9501245-B2
Application number	US-201414279712-A
Country	US
Kind code	B2
Filing date	May 16, 2014
Priority date	May 2, 2014
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

A new approach is proposed that contemplates systems and methods to virtualize a physical NVMe controller associated with a computing device or host so that every virtual machine running on the host can have its own dedicated virtual NVMe controller. First, a plurality of virtual NVMe controllers are created on a single physical NVMe controller, which is associated with one or more storage devices. Once created, the plurality of virtual NVMe controllers are provided to VMs running on the host in place of the single physical NVMe controller attached to the host, and each of the virtual NVMe controllers organizes the storage units to be accessed by its corresponding VM as a logical volume. As a result, each of the VMs running on the host has its own namespace(s) and can access its storage devices directly through its own virtual NVMe controller.

First claim

What is claimed is:

1. A system to support non-volatile memory express (NVMe) controller virtualization, comprising:
an NVMe managing engine running on a host, which in operation, is configured to
create and initialize one or more virtual NVMe controllers running on a single physical NVMe controller, wherein each of the virtual NVMe controllers is configured to support one of a plurality of virtual machines (VMs) running on the host to access its storage units;
subsequently add or remove one or more of the virtual NVMe controllers based on the number of VMs running on the host and physical limitations of the physical NVMe controller to support the virtual NVMe controllers;

said virtual NVMe controllers running on the single physical NVMe controller, wherein each of the virtual NVMe controllers is configured to:
establish a logical volume of storage units having one or more corresponding namespaces for one of the VMs to access;
retrieve and process commands and/or data from the VM to access the namespaces or the logical volume of storage units for the VM;
provide processing results of the commands and/or data back to the VM via the virtual NVMe controller.

2. The system of claim 1 , wherein:
the host of the VMs is an x86/ARM server.

3. The system of claim 1 , wherein:
each of the virtual NVMe controllers is a hardware accelerated software engine emulating functionalities of an NVMe controller.

4. The system of claim 1 , wherein:
each of the virtual NVMe controllers is configured to interact with and allow access from only one of the VMs.

5. The system of claim 1 , wherein:
each of the storage units in the logical volume is attached to the physical NVMe controller locally.

6. The system of claim 1 , wherein:
each of the namespaces is organized as a plurality of directories and files to enable the VM to access the logical volume via a filename or a path in the namespace on a storage device.

7. The system of claim 1 , wherein:
multiple of the plurality of VMs are enabled to access the same logical volume and each logical volume is enabled to be shared among the multiple VMs through their respective virtual NVMe controllers.

8. The system of claim 1 , wherein:
each of the virtual NVMe controllers is configured to support identity-based authentication and access from its corresponding VM for its operations, wherein each identity permits a different set of API calls for different types of commands used to create, initialize and manage the virtual NVMe controller and/or provide access to the storage units for the VM.

9. The system of claim 1 , wherein:
the VMs running on the host are isolated from each other so that one VM cannot access the data storage and/or communication of any other VMs running on the same host.

10. The system of claim 1 , wherein:
the NVMe managing engine is a physical function (PF) NVMe driver embedded in a hypervisor running on the host.

11. The system of claim 1 , wherein:
the NVMe managing engine is configured to determine the number of the virtual NVMe controllers needed to support the VMs, load drivers to the physical NVMe controller used to communicate with the VMs, launch and monitor the virtual NVMe controllers dedicated to serve the VMs.

12. The system of claim 1 , wherein:
the NVMe managing engine is configured to freeze the virtual NVMe controller assigned to the VM in time and to move the virtual NVMe controller to a different physical NVMe controller as part of the migration process for the VM, which is being migrated from the current host to another host.

13. The system of claim 1 , wherein:
the physical NVMe controller connects to the host via a Peripheral Component Interconnect Express (PCIe)/NVMe link.

14. The system of claim 13 , wherein:
each of the VMs running on the host has an NVMe driver configured to interact with the physical NVMe controller and the virtual NVMe controllers via the PCIe/NVMe connection.

15. The system of claim 1 , wherein:
the physical NVMe controller provides both Physical Functions (PFs) and Virtual Functions (VFs) to support the virtual NVMe controllers running on it.

16. The system of claim 15 , wherein:
each of the virtual NVMe controllers is configured to process the commands of its corresponding VM by invoking the supporting VF functions provided by the physical NVMe controller.

17. The system of claim 1 , wherein:
the physical NVMe controller further includes:
an NVMe queue manager (NQM) configured to manage a plurality of queues that include the commands from the VM;
a processing engine that includes at least a memory unit and a processor configured to process the commands in the queues.

18. The system of claim 17 , wherein:
the NQM fetches the commands from a first queue, provides the commands to the processing engine for processing, and puts the processing results of the commands in a second queue for the VM.

19. A computer-implemented method to support non-volatile memory express (NVMe) controller virtualization, comprising:
creating and initializing one or more virtual NVMe controllers running on a single physical NVMe controller, wherein each of the virtual NVMe controllers is configured to support one of a plurality of virtual machines (VMs) running on a host;
subsequently adding or removing one or more of the virtual NVMe controllers based on the number of VMs running on the host and physical limitations of the physical NVMe controller to support the virtual NVMe controllers;
establishing a logical volume of storage units having one or more corresponding namespaces for each of the VMs to access via its corresponding virtual NVMe controller;
retrieving and processing commands and/or data from each of the VMs by its corresponding virtual NVMe controller to access the namespaces or the logical volume of storage units for the VM;
providing processing results of the commands and/or data back to the VM via its corresponding virtual NVMe controller.

20. The method of claim 19 , further comprising:
connecting the physical NVMe controller to the host via a Peripheral Component Interconnect Express (PCIe)/NVMe link.

21. The method of claim 19 , further comprising:
fetching the commands from a first queue, providing the commands to a processing engine for processing, and putting the processing results of the commands in a second queue for the VM.

22. The method of claim 19 , further comprising:
organizing each of the namespaces as a plurality of directories and files to enable the VM to access the logical volume via a filename or a path in the namespace on a storage device.

23. The method of claim 19 , further comprising:
enabling multiple of the plurality of VMs to access the same logical volume and each logical volume to be shared among the multiple VMs through their respective virtual NVMe controllers.

24. The method of claim 19 , further comprising:
supporting identity-based authentication and access by each of the virtual NVMe controllers from its corresponding VM for its operations, wherein each identity permits a different set of API calls for different types of commands used to create, initialize and manage the virtual NVMe controller and/or provide access to the storage units for the VM.

25. The method of claim 19 , further comprising:
isolating the VMs running on the host from each other so that one VM cannot access the data storage and/or communication of any other VMs ru

Assignees

• Cavium Inc

Inventors

• Hussain Muhammad Raghib
• Murgai Vishal
• Panicker Manojkumar
• Masood Faisal
• Folsom Brian
• Kessler Richard Eugene

Classifications

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0655

  Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/0664

  at device level, e.g. emulation of a storage device or system · CPC title

• G06F3/0688

  Non-volatile semiconductor memory arrays · CPC title