close window

Print print

IBM Releases PowerKVM Hypervisor for POWER8 Systems

In June 2014, IBM released the PowerKVM hypervisor for POWER8* scale-out systems. KVM, which stands for Kernel-based Virtual Machine, is an industry-standard hypervisor that has gained significant popularity in the open-source space. Like other hypervisors, KVM enables the sharing of real compute, memory and I/O resources through server virtualization.

According to the OpenStack User Survey conducted in November 2014, KVM has 87 percent of the OpenStack cloud marketplace (bit.ly/1gZQQAK). When PowerKVM was released, the Ubuntu distribution of Linux* was also released for Power Systems*. Ubuntu enjoys 64 percent of the OpenStack cloud market, according to the same survey, and has 54 percent of the Amazon Cloud space (thecloudmarket.com/stats). The two releases provide a great one-two punch for using Linux on Power* in the cloud. While the KVM hypervisor has 87 percent of the hypervisor market share in the cloud, Ubuntu is the leading OS for cloud-based solutions with 64 percent of the market share.

The Platform Decision

IT implementation decisions have always centered on the platform; however, the definition of platform has seen a number of iterations. Figure 1 shows a typical stack of software components.

The requirements of the application to be implemented used to drive the decision and dictate the remaining portions of the stack. Over time, though, the same applications have become available across multiple OSs, which have become available across multiple hypervisors such that the platform decision is driven down to the hardware architecture. Because of this, Power Systems architecture is the clear winner. Support of PowerKVM means those shops that have implemented applications on KVM on hardware other than Power now have the option of implementing on Power with the only change being the hardware platform itself.

Design and Implementation

With that background, let’s examine the features of the KVM hypervisor. KVM provides additional capability for existing Linux distributions. KVM takes advantage of the modular design of Linux to deliver virtualization via a kernel module (kvm.ko) that adds core virtualization and hypervisor features to the Linux kernel. In addition to the kernel module, a user-space program called QEmu provides emulation (not used in PowerKVM) as well as virtual devices and control mechanisms. Rounding out the virtualization functionality is a library interface called libvirt, which uses a standard library to manage virtual machines (VMs). Those three pieces (kvm.ko, QEmu, libvirt) take a Linux kernel and turn it into a hypervisor. Rather than replacing existing functionality, such as scheduler and memory management, KVM uses the existing Linux facilities so the KVM developers can concentrate on hypervisor and virtualization functionality while other open-source developers can concentrate on their areas of specialty.

In the KVM space, VMs exist as user-space processes and are referred to as guests while the Linux kernel with the KVM module is referred to as the host.

What Is Paravirtualization and Why Is It Important?

KVM requires hardware virtualization extensions, which of course are available on the Power platform, as well as paravirtualization where applicable. A high-level understanding of paravirtualization is important to an understanding of KVM on the Power platform and the capability to leverage the same Linux distributions on Power regardless of the hypervisor employed. Paravirtualization presents a software interface to VMs that’s similar but not identical to that of the underlying hardware. In a paravirtualization environment, the guests/VMs/LPARs are aware they’re running in a virtualized environment and as a result have knowledge of when to call into the hypervisor. This means the hypervisor has no need to trap and emulate privileged instructions and allows the guests to run at full hardware speed.

The Power platform consists of a virtual integration of hardware, firmware and software components with standards, guidelines and specifications that are established by the power.org governing body. Power.org definitions include the following:

  • Processor ISA
  • Memory management
  • Architecture platform reference specifications

The definition is referred to as the POWER Architecture Platform Reference (PAPR) and it also describes a variety of VM functions including:

  • Bootstrap
  • Runtime
  • Shutdown function
  • Virtualization operation

These virtualization standards are implemented using a combination of hardware, firmware and software. Figure 2 provides a high-level view of the use of the PAPR specification under the PowerVM* hypervisor. Communication between the VMs and the hypervisor occurs through the partition firmware (open firmware) to the hypervisor with the communication adhering to the PAPR specification. Virtualization on Power provides for the cooperation of hardware, firmware and software, and allows for efficient management of privileged hardware resources. The hardware platform provides for three privilege levels identified as hypervisor, supervisor and user. The hypervisor state includes partitioning and virtualization facilities via special purpose registers including an MMU hash table and interrupt control. The entire Power platform is designed with paravirtualization (the cooperation of VMs and hypervisor) in mind where certain aspects of the system can’t be emulated or spoofed and the VMs have certain virtualization responsibilities, which allow them to call directly into the hypervisor via hcalls.

As long as the OS running in the VM is paravirtualized, it can run on the Power system, which has always been paravirtualized. While the hypervisor runs in hypervisor mode and has access to all memory and system resources, the OS runs in supervisor mode. This is true regardless of whether the OS running in the VM is Linux, IBM i or AIX*.

Now, let’s look at the KVM version of virtualization on Power in Figure 3. A comparison between the two approaches shows the following differences:

  1. A new partition firmware is employed via the Slim-Line Open Firmware (SLOF).
  2. Communication with the hypervisor (PowerKVM) occurs via a QEMU process that runs in the host (hypervisor) for each guest (VM) running on the system.
  3. Interaction between the guest OS and the hypervisor (qemu process) adheres to the PAPR specification.
  4. No VIOS is present; instead, I/O virtualization is provided through the QEMU process of each guest running in the KVM environment.

Because the interaction between the guest OSs and the hypervisor adheres to the PAPR specification, two benefits occur:

  1. Linux OSs that are compiled for PPC64 will run on either the PowerVM or PowerKVM hypervisors.
  2. Adhering to the PAPR specification means that the guest OSs are paravirtualized; therefore they run at hardware speed.

What’s the Difference Between the Two Hypervisors?

At this point, you might be wondering why KVM is being supported on the Power platform. The PowerVM hypervisor, which has been available since 2004, supports the virtualization of processors, memory, storage and networking for Linux as well as IBM i and AIX environments. PowerKVM, on the other hand, only supports Linux and is intended for clients that have Linux-centric administrators. Typically, KVM will be used on the Power platform by clients who already have experience with KVM on other hardware platforms, such as x86-based systems.

The feature set between the PowerVM hypervisor and PowerKVM differs in a number of areas. For example, the PowerVM hypervisor supports dedicated as well as shared processors, processor pools, guaranteed entitlement, hard capping, and capacity on demand. PowerKVM on the other hand allocates processors (from a definition viewpoint) in full processor increments, and the hypervisor determines when and for how long the QEMU process that represents the guest OS is dispatched on the processor(s). For I/O support, the PowerVM hypervisor supports NPIV, SR-IOV, dedicated I/O devices as well as redundant I/O virtualization (via dual VIOS partitions) while PowerKVM only currently supports virtualized I/O devices that are provided by the QEmu process of each guest. The PowerVM hypervisor also has support for dynamic LPAR, system pools as well as non-Linux operating systems (i.e., AIX and IBM i). Keep in mind these statements represent the current feature sets and may change with later releases.

Microthreading

The unique features of PowerKVM include support for NFS and iSCSI storage as well as exploitation of the microthreading capabilities of the POWER8 processor. Under the traditional PowerVM and PowerKVM model, a complete core is dispatched to the VM regardless of the workload’s entitlement. This means is that if a workload has an entitlement of .20 processors than for 2 milliseconds, the core is dispatched in totality to the workload. So every time a workload has exhausted its entitled capacity, a context switch occurs to move one workload off the core to be dispatched to the next workload.

With microthreading, the hypervisor (in this case, PowerKVM) can dispatch multiple workloads (VMs) across a core at the same time by leveraging the SMT capabilities of the Power processor.

Currently PowerKVM can dispatch up to four VMs/guests on a single core and is enabled via the ppc64_cpu command. The microthreading capability has the potential to provide better performance for multiple small workloads because it eliminates the need to context switch between the workloads.

Adding More Powerful Tools

PowerKVM continues IBM’s long-standing support of open-source initiatives by bringing the open-source hypervisor to the Power Systems environment. It represents an exciting addition to the Power ecosystem and enables Linux-centric shops to leverage the benefits of Power Systems while using a hypervisor that cuts across multiple hardware platforms.