How to Migrate to SR-IOV
James Chandler explains how to leverage this new Lab Services I/O tool.
By James Chandler06/01/2020
Q: What is SR-IOV and how do I migrate to it?
Single-Root I/O Virtualization (SR-IOV) was first introduced for IBM POWER8® servers in 2015. SR-IOV is an industry-standard extension to the PCI Express specification that allows multiple client partitions to share a physical adapter. SR-IOV is similar to shared-pool processor-sharing, in that each client LPAR is guaranteed a “slice” of the physical adapter port’s capacity. Adopting SR-IOV can result in significant CPU savings, increased network bandwidth and reduced latency.
SR-IOV was first introduced using native logical ports, where each port is mapped to an underlying SR-IOV physical adapter. SR-IOV logical ports take advantage of direct access I/O, which provides CPU and latency characteristics similar to a dedicated adapter. With direct access I/O, no additional hypervisor or VIOS overhead is required to process client network traffic as compared to a traditional virtual Ethernet configuration. This can result in a significant reduction in CPU consumption, as well as reduced latency for a given workload. One drawback of SR-IOV logical ports is that they don’t support the IBM Live Partition Mobility (LPM) feature.
In late 2018, IBM introduced SR-IOV VNIC (Virtual Network Interface Controller), which supports LPM. Unlike SR-IOV logical ports, which have no dependency on a VIO server, VNIC utilizes a VF device that’s automatically created and assigned to the VIOS when the VNIC client adapter is created. While VNIC doesn’t support the direct access I/O model of SR-IOV logical ports, it does still provide a reduction in VIOS CPU and memory consumption, as well as lower latencies when compared to SEA configurations with virtual Ethernet.
Why Transition to SR-IOV?
In addition to the savings in CPU and memory consumption, SR-IOV can provide a significant increase in throughput for workloads that require a large number of transactions per second (TPS). There are real-world scenarios where SR-IOV logical ports and VNIC can satisfy a client’s network performance requirements that wouldn’t have been possible with previous network virtualization technologies (see Figure 1, below).
Additionally, SR-IOV provides better out-of-the-box performance than virtual Ethernet, with no additional tuning.
How Do I Convert From Virtual Ethernet to SR-IOV VNIC?
With a traditional virtual Ethernet configuration utilizing a shared Ethernet adapter, each client LPAR would be assigned an adapter through the client’s HMC partition profile. The client OS will have a number of logical devices associated with the virtual Ethernet adapter, possibly with their own interface-specific tunables.
Migrating from virtual Ethernet to SR-IOV would require (at a minimum): adding a new SR-IOV adapter for each individual client LPAR, migrating the IP address, re-applying any interface-specific tunables, cleaning up OS devices and saving the changes to the client’s partition profile. This can be a very time-consuming and error-prone process, especially for clients that have a large number of partitions to migrate to SR-IOV.
What’s the IBM SR-IOV Migration Tool?
IBM developed the SR-IOV Migration Tool to assist clients in the migration process from virtual Ethernet to SR-IOV. With the IBM SR-IOV Migration Tool, clients can dynamically and non-disruptively migrate to SR-IOV VNIC with no downtime. This includes live-migrating the IP address from the old virtual Ethernet device to the new SR-IOV VNIC adapter (AIX® only). The tool also allows for the optional removal of any OS-level devices, as well as removing the virtual Ethernet adapter from the client’s HMC partition profile.
The tool can also gather information about the client’s existing SR-IOV adapters and their configuration, such as the number of logical ports assigned to a given physical port, port labels and capacities, and report that information in an easily digestible format. This information can be invaluable when planning for the adoption of SR-IOV, as it allows the client to see how many logical ports are available, as well as their remaining capacity.
The tool currently supports IBM AIX versions 7.1 and 7.2, with SUSE Linux® on POWER® support planned in the near future. Additionally, the tool allows for adjusting the maximum capacity setting for a given VNIC device, a feature that’s not available through the HMC GUI. The SR-IOV Migration Tool can be a very valuable asset, allowing clients to implement a repeatable, consistent methodology for the adoption of SR-IOV in their environment.
Contact IBM Systems Lab Services to learn more.
James Chandler is a certified IT management consultant working for the IBM Systems Lab Services Power Systems Delivery Practice.