Monitor Your IBM i Application and System Health
Monitor your IBM i applications to mazimize performance.
By Dawn May11/02/2020
Using high availability solutions, you can build a more reliable IBM i infrastructure. But no matter how robust your technical solution may be, the potential always exists for unexpected situations. Application or system defects may surface, usage patterns may change, workloads may grow or the number of users may increase. These changes can add unanticipated stress to the system. Simple human error may also result in unfortunate surprises.
Unexpected situations can be costly. Outages mean increased expenses to identify and resolve the problem, as well as lost revenues. Outages may also irritate clients, which can threaten the viability of your business.
Why Monitor Systems and Applications?
For these reasons, you must monitor your system and applications on a regular basis to be able to identify potential problems before they escalate into real problems.
In my consulting practice, I am often contacted when systems or applications are not working optimally. Typical complaints involve performance concerns, but as I review system environments, I almost always discover
a variety of preventable issues easily solved by proactive monitoring.
What things must you consider when developing a monitoring strategy?
What should you monitor?
First of all, determine what you need to monitor. Some metrics must always be monitored, such as CPU, memory and disk utilization. It’s also important to monitor the number of jobs and spooled files in the system.
Metrics unique to your environment or applications offer an unlimited number of potential data points to track. The hard part is identifying which metrics are essential to monitor in your environment. Keep in mind, that a monitoring solution isn’t static; you must continually review what you monitor and watch the overhead incurred by the monitoring solution itself.
What automation is needed?
Next consider what type of automation you need in your solution. Without automation, you don’t have a true monitoring solution. You want the system and application monitoring to automatically alert you when something is not as expected. The type of notification can be simple, such as a message to QSYSOPR, in which case you need to have a message monitoring solution in place.
Other notification options are also available. Some shops prefer email notifications, others want text messages for immediate awareness. Some companies even feed information into a collaboration app channel, such as Slack, to alert their teams of unexpected situations.
Whatever approach you select, it needs to be a good fit for what you are monitoring and timely awareness of any issues.
Which Tools are a Fit?
A variety of monitoring solutions are available in the IBM i marketplace, with a range of costs and complexity. I assert that simplicity is essential in a monitoring solution. I’ve found that adding additional complexity to monitor your system can have a negative impact. The simplest monitoring tools for IBM i are the free ones that ship with the OS, but they are also the most basic.
Let’s review what comes with the OS so you can get started “with limited expenses” as you consider a monitoring strategy.
Monitor performance with system monitors
Understanding your system performance characteristics is essential, whether your shop is small or large, simple or complex. Knowing your performance signature makes it easier to identify what’s different when something goes wrong. You can manually review your performance signature using the Performance Data Investigator, but automated monitoring lets you define metrics and thresholds ahead of time so that the system notifies you about potential issues as they happen.
System monitors are part of Navigator for i. They allow you to select key metrics in your environment and identify and set thresholds in which to get notified about potential problems. The automation can be a simple command (e.g., SNDSMTPEMM). You can also write a program to build whatever type of automation your shop requires.
System monitors support a variety of metrics you can monitor. Metrics that I generally recommend for monitoring include:
- CPU utilization
- Machine and user pool faulting rates
- Spooled file creation rate
- Temporary storage utilization
- Disk read and write response times
- HTTP server requests and response
System monitors also allow for real-time visualization of the monitor data, as Figure 1 (click here) shows.
Real-time monitoring is insufficient; you also must track your performance trends over time. Graph History, introduced in 7.3, makes it easy.
Monitor for system limits
While IBM i is extremely scalable and can support very complex applications, there are limits within the system that, if hit, will likely cause an application or even system outage. IBM documents these system limits in the maximum capacities section in the Knowledge Center.
Documenting these limits is nice, but IBM also provides System Health Services that can be used to understand how close you are to critical system limits. You can create simple queries to determine how close you are to the various limits and review the limits over time. You can add triggers to the system limits table for notifications if a limit is exceeding its threshold of concern.
Some example limits to consider monitoring include:
- Maximum number of jobs or spooled files in the system
- Maximum number of rows or deleted rows in a partition
- Maximum size of an IBM Db2® table
- Largest files in the IFS
Application and environment consistency
You can use the Administration Runtime Expert (ARE) to ensure that critical attributes in your environment and application are not changed. You first must identify the key attributes, then you can create templates that define the attributes with the expected values. You then perform a verification with that template against the selected systems to check on the actual attributes as compared to the expected attributes.
ARE can be used to ensure that application and environment attributes are consistent across multiple partitions or to compare production versus test environments. ARE verifications can be scheduled with email notifications sent if errors are found during the verification. Simple summary reports, such as the example in Figure 2 (click here), make it easy to identify the inconsistencies.
If you're not monitoring your system and applications today, get started with the simple and free tools included with IBM i; as your monitoring needs grow, you can explore third-party solutions.
They key thing is to do it. Even though your environment may be running smoothly now, that does not mean it always will.
for Monitoring IBM i
• You can only monitor a single partition with Navigator System Monitors. If you have multiple partitions, IBM provides a “copy monitor” function, which allows you to copy a monitor from one partition to another. The partition to which you wish to copy the monitor must be added to the Target Systems on the original partition.
• Navigator for i provides a basic message monitoring solution with message monitors. You may already have a message monitoring solution from an IBM i partner, but if you don’t, message monitors are an easy way to get started.
• If you have a need to monitor for a message sent to a job log, watches are the way to go. Watches have low overhead and allow you to efficiently monitor for a message sent to any or all job logs.
• You must enable historical data collection before you can visually review that data with the “graph history” function.
• You can get notified when a row is added to or deleted from the system limits table by adding an AFTER_INSERT or AFTER_DELETE trigger. This allows you to send a notification of some sort when a change to a tracked system limit has been logged.
• The IBM Administration Runtime Expert (ARE), as shipped by IBM, provides a rich set of attributes you can verify. For instance, you may need to automatically set attributes based on runtime workloads. With ARE, you can create custom plugins that can do any type of verification.
Dawn May is an IBM i consultant. She owns Dawn May Consulting, LLC in the Greater Boston area. Dawn is a former IBM senior technical staff member.