Engineered Together—IBM Java and zEC12 Boost Workload Performance

Large Pages Frames

Processor time spent converting virtual addresses into physical real memory addresses slows down application execution times. Modern computers perform this conversion for each page frame. This conversion information is cached in Translation Lookaside Buffers (TLBs). With 64-bit Java applications using significantly larger Java heaps, it has become apparent that a larger page-frame size is required to reduce the processor conversion overhead of virtual addresses to real memory addresses.

Historically, System z has had a 4K (4096-byte) page frame. The IBM System z10 mainframe introduced 1-megabyte large pages (LPs). A 1MB LP contains the equivalent of 256 4K pages. Starting in z/OS V1.9 and 64-bit Java V6, 1MB fixed (non-pageable) LPs were supported on IBM System z10. The following configuration steps were required to enable them:

  1. IEASYSxx keyword LFAREA=xxxG or LFAREA=xxxx% to specify number of system-wide 1MB LPs to be reserved during z/OS initial program load (IPL)
  2. Enable unauthorized callers with RACF class named IARRSM.LRPAGES
  3. 64-bit Java with option –Xlp to use 1MB fixed non-pageable LPs for entire Java heap

To use 1MB fixed non-pageable LPs, system administrators had to perform Step 1 by calculating and specifying the system-wide total number of 1MB fixed non-pageable LPs required for all Java Virtual Machines and IBM WebSphere Regions at system IPL time. They also had to perform Step 2 for unauthorized users to use 1MB fixed non-pageable LPs. When the JVM is initially started with –Xlp, it attempts to allocate the entire Java heap with 1MB fixed non-pageable LPs. If there aren’t enough of these LPs, Java will allocate 4K pages for the entire Java heap.

To simplify and improve the consumability of 1MB large pages, zEC12 introduced the Flash Express feature. The z/OS V1.13 RSM enablement offering Web-deliverable allowed for 1MB pageable LPs. These are only supported on zEC12 with the Flash Express feature. Java 6.0.1 SR4 and Java 7 SR3 are the first Java releases to support z/OS 1MB pageable LPs. They have several advantages over their fixed, nonpageable counterparts, such as:

  • Calculating and specifying LFAREA is not required.
  • They’re allocated only if they’re actually used on first reference.
  • 1MB LPs are pageable to and from zEC12 flash memory.
  • Any unauthorized callers may use them without requiring RACF permissions.
  • 31-bit and 64-bit Java may use them.
  • They may be used for both Java object heap and Java machine hardware instructions (JIT code cache) with: -Xlp:objectheap:pagesize=1m,pageable and -Xlp:codecache:pagesize=1m,pageable.

Performance measurements of Java multi-threaded and WebSphere Day Trader 2.0 benchmarks have shown approximately a 5 percent throughput improvement using either 1MB fixed non-pageable or 1MB pageable LPs versus 4K pages. In addition to object-heap, 1MB pageable LPs saw another 1 percent improvement in the Java multi-threaded benchmark when 1MB JIT code cache LPs were also enabled using:


The following provides information on what type of LPs are available for use on the current system and what page sizes were used by Java:


In the event that z/OS runs out of 1MB pageable LPs, some of the Java pages may contain both 1MB and 4K pageable pages. The introduction of 1MB pageable LPs using the zEC12 Flash Express feature makes it significantly easier to take advantage of large pages and improve performance.

Performance Results

To illustrate how these software and hardware innovations work together to increase throughput for various Java workloads, consider the following benchmark results illustrated in figures 5, 6 and 7:

Figure 5 shows the results for a multi-threaded Java application. A 45 percent hardware improvement was measured for SDK 7 SR1 on zEC12 versus SDK 7 SR1 on zEnterprise 196. An additional 13 percent improvement was measured using SDK 7 SR3 with options –Xaggressive to enable new zEC12 instruction exploitation and –Xlp:codecache:pagesize=1m,pageable to enable 1MB pageable LPs. An aggregate throughput performance improvement of 60 percent was measured.

Figure 6 demonstrates the commitment IBM has made to improving Java applications by as much as 12 times over multiple generations of System z hardware and Java software releases. Similar improvements have been measured on Linux for System z.

Figure 7 shows the improvements to the WebSphere Liberty profile from zEC12 and Java 7 SR3 on z/OS. zEC12 provides a 28 percent improvement in Servlet and JSP throughput. Java 7 SR3 (with LPs and –Xaggressive) provides an additional 7 percent boost, resulting in an aggregate improvement of 37 percent. When including enhancements provided by the WebSphere Application Server (WAS) Liberty profile over the traditional WAS environment, an aggregate 83 percent increase in throughput for Sevlet and JSP serving was measured.

More to Come

IBM continues to invest aggressively in the vertical integration of its Java technology on System z. This is highlighted by innovative new hardware features, such as transactional memory and pageable 1MB LPs using Flash Express technology, which are meant to position the platform for trends in increased thread counts and growing application footprints.

The zEC12 touts a new core design well suited for Java-based workloads. The JVM continues to charge forward with a roadmap that has demonstrated deep exploitation of new hardware and impressive release-to-release performance enhancements. The latest installment includes a 45 percent hardware-based improvement from zEC12, and an additional 13 percent improvement from the Java 7 SR3 when enabling –Xaggressive and pageable LP support.

Note that no modification of Java source code or recompilation is required to take advantage of these features. IBM plans to further exploit the TX facility and other zEC12 features, including as 2GB large pages and the Runtime Instrumentation facility in future releases of JVM. Stay tuned!

Clark Goodrich is a senior software engineer in the IBM Systems, IBM Z performance organization working on the Java compiler. He has been with IBM since 1978 and holds six IBM patents. Prior to joining IBM, he did real-time programming at NASA's Goddard Space Flight Center for the Space Shuttle.

Jerry Zheng is a staff software developer in Java Just-in-Time Compiler development who is leading the effort of supporting transactional memory in IBM J9 JVM. He joined IBM in 2008 and has been working on exploiting new System z hardware features in a dynamic compiler.

Marcel Mitran is a Distinguished Engineer and Chief Technology Officer for IBM LinuxONE. He is based out of the IBM Toronto Lab. Marcel has spent more than 15 years developing hardware and software technology to solve problems of the modern enterprise.

comments powered by Disqus



2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.


Your Input Needed: IBM Systems Media Reader Survey

Educated for Success

Marist College students benefit from school's partnership with IBM

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
Mainframe News Sign Up Today! Past News Letters