IBM i > ADMINISTRATOR > SYSTEMS MANAGEMENT

PCIe Gen3 I/O Expansion Drawer Overview


IBM recently announced the availability of PCIe Gen3 I/O expansion drawers for use with POWER8 scale-out servers. This is an important addition for these servers because the number of internal server adapter slots is limited and dependent on the number of installed processor modules. Without this I/O expansion drawer, sometimes a second processor module had to be configured to enable more internal adapter slots to support a desired mix of adapter cards. The I/O expansion drawer lets additional adapter slots be configured without a second processor module. IBM also introduced more flexible configuration I/O expansion drawer configuration options for the POWER8 Enterprise servers. This article provides an overview of the I/O expansion drawer along with configuration details.

PCIe Gen3 I/O Expansion Drawer Highlights

The I/O expansion drawer can now be configured with all of the POWER8 server models. The I/O expansion drawer supports up to 12 PCIe Gen3 adapter slots. Note that the I/O expansion drawer only provides adapter slots and doesn’t offer disk bays. Local disk can be configured using the EXP24S SAS disk drawer along with one or more SAS adapters.

I/O Drawer Components

Four components make up a functioning I/O drawer configuration:

  • PCIe Gen3 I/O Expansion Drawer
  • PCIe3 6-Slot Fanout Module
  • PCIe3 Optical Cable Adapter
  • Optical Cable Pair

These components are shown in Figure 1. The PCIe Gen3 I/O expansion drawer is an empty, 4U chassis configured with two redundant power supplies. Inside the I/O expansion drawer is space where one or two 6-slot fanout modules can be inserted. Each 6-slot fanout module has six PCIe Gen3 adapter cards slots: Four are type x8 and two are type x16. All of the adapter slots are hot-pluggable and utilize full-height adapter cards. Each 6-slot fanout module is connected to its own PCIe3 optical cable adapter, which is installed in the host server. For each 6-slot fanout module that’s configured, you actually gain five useable slots because the optical cable adapter consumes one slot in the host server. Special optical cables are used to connect the 6-slot fanout module to the optical cable adapter. The optical cables are available in 2m, 3m and 10m lengths. Cable length availability varies with the host server model.

Configuration Details

Following is a summary of the various I/O expansion drawer components and their IBM feature codes. Note that the optical cable adapter and optical cable pair features vary depending on the server model.

  • #EMX0 - PCIe Gen3 I/O Expansion Drawer (all server models)
  • #EMXF - PCIe3 6-Slot Fanout Module for PCIe3 Expansion Drawer (all server models)
  • #EJ08 - PCIe3 Optical Cable Adapter for 4U CEC (4U Scale-Out, E850)
  • #EJ05 - PCIe3 Optical Cable Adapter for PCIe3 Expansion Drawer (2U Scale-Out)
  • #EJ07 - PCIe3 Optical Cable Adapter for PCIe3 Expansion Drawer (E870/E880)
  • #ECC6 - 2M Optical Cable Pair for PCIe3 Expansion Drawer (E870/E880)
  • #ECC7 - 3M Optical Cable Pair for PCIe3 Expansion Drawer (Scale-out and E850)
  • #ECC8 - 10M Optical Cable Pair for PCIe3 Expansion Drawer (E850/E870/E880)
  • #ECC9 - 20M Optical Cable Pair for PCIe3 Expansion Drawer (E870/E880)

Following is a sample listing of features when adding one expansion drawer with two 6-slot fanout modules to a Model E870 server:

  • Qty 1 - #EMX0 PCIe Gen3 I/O Expansion Drawer
  • Qty 2 - #EMXF PCIe3 6-Slot Fanout Module for PCIe3 Expansion Drawer
  • Qty 2 - #EJ07 PCIe3 Optical Cable Adapter for PCIe3 Expansion Drawer
  • Qty 2 - #ECC6 2M Optical Cable Pair for PCIe3 Expansion Drawer
  • Qty 2 – Power cord of appropriate length

Maximum Quantity of I/O Drawers by POWER8 Model

The number of supported I/O drawers is related to the number of POWER8 processor modules installed in the server. Each POWER8 processor module features two integrated PCIe Gen3 controllers. Configuring one optical cable adapter per PCIe Gen3 controller allows for two 6-slot fanout modules for each installed processor module, or more simply, one I/O expansion drawer per installed processor module. This rule of thumb holds true for all POWER8 server models, except the 2-socket 822L Linux only model, which supports just one expansion drawer. Table 1 shows the maximum number of I/O drawers allowed per server model. The I/O drawers can be configured with a single 6-slot fanout module. The second 6-slot fanout module can be added later. The host server must be powered off when adding or removing the 6-slot fanout modules to a server.

New PCIe Gen3 and Other Popular Adapter Cards

IBM also announced a new, 4-port 10Gb Ethernet adapter. The most commonly configured network adapters include the following:

  • 4-port 1Gb Ethernet
  • 4-port Ethernet (two 10GbE and two 1GbE ports)
  • 2-port 10Gb Ethernet
  • 4-port 10Gb Ethernet

Common SAN attachment adapters include:

  • 2-port 4Gb FC
  • 2-port 8Gb FC
  • 4-port 8Gb FC
  • 2-port 16Gb FC

Other adapter cards are available including SAS and USB.

Flexible Configuration Options

The option to configure PCIe Gen3 I/O expansion drawers for use with the POWER8 scale-out servers has significantly improved design flexibility when deploying these servers. Single processor module based systems can now be configured with up to 16 useable adapter slots. In addition, the enhanced configuration options for the E870 and E880 Enterprise servers allows these servers to be configured with 6 to 48 expansion slots per system node.

Charlie Cler supports customers in a solutions-architect role at Forsythe Technology Inc. He can be reached at ccler@forsythe.com.



Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.


comments powered by Disqus

Advertisement

Advertisement

2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

IBM i > ADMINISTRATOR > SYSTEMS MANAGEMENT

Going Mobile With DB2 Web Query

IBM i > ADMINISTRATOR > SYSTEMS MANAGEMENT

Directing i

How to enable IBM i for management by IBM Systems Director

IBM i > ADMINISTRATOR > SYSTEMS MANAGEMENT

Putting the "V" in Virtualization

IBM eServer line delivers on the promise of virtualization

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
IBMi News Sign Up Today! Past News Letters