MAINFRAME > Business Strategy > BI and Analytics

Integrate z Systems Operational Data Into Your Existing Analytics Solution

IT Operations Analytics

More clients are recognizing the value of IT Operations Analytics (ITOA) to reduce their root cause analysis elapsed time, to improve the efficiency of overall IT and to prevent outages. Initial exploitations of ITOA were limited to the analysis of the data coming from distributed platforms, with the possible addition of network logs and performance data. However, many applications (i.e., the most business-critical ones), behind a nice and user-friendly web interface and some part running on distributed platforms, rely on the mainframe-backend applications and systems like CICS and DB2 on z/OS.

A complete ITOA solution can’t miss mainframe operational data, and it needs to be stored in the same analytic engine as the data coming from the distributed platforms amd network in order to allow an end-to-end view of the IT systems and applications. While in distributed platforms and networks there are many applications able to retrieve log data, the know-how needed to timely retrieve important mainframe operations data is Important.

Single Point for Data

IBM recently released IBM Common Data Provider for z Systems (CDPz), a single collection point to access near real-time operations data like SMF records, RMF data and a wide variety of log data on z Systems. This product is based on 30+ years’ experience in collecting, parsing and analyzing z Systems operational data. CDPz is able to collect z Systems IT operations data from more than 140 data sources, including 100+ SMF record types as well as application logs (e.g., WebSphere, CICS) or generic files, and allows access of analytics data within minutes, collecting the requested data once and providing multiple different end users and analytics products with the data they need.

CDPz offers advanced filtering capabilities to target specific use cases and reduce data volumes and network traffic. Batch data collection is available to control CPU consumption spikes. All of the configuration of the specific data to be collected, the transformation to be applied and the target to be fed can be easily defined in the web user interface provided with the product as a z/OS Management Facility plugin (see Figure 1). CDPz offers the unique capability to filter the log data based on the content of the data itself. The filtering is based on a regular expression defined by the user, who can decide if the log records that match the regular expression defined are to be sent or discarded. Leveraging this feature, it’s possible to satisfy the need to send, for example, only the logs related to as specific application to the data lake accessible from the specific application development team. It’s useful to have the right level of data segregation, allowing the access to the data only if required. This feature is crucial to enable a complete DevOps paradigm, where the development team can have all the logs from mainframe and distributed in the same place, with no IBM z expertise.

Another important characteristic of CDPz is the extensibility of the product in terms of data sources. With Open Streaming API, it’s possible to leverage CDPz to send user application data to a consumer defined through the same web user interface. REXX and Java APIs are provided with the product to enable this feature. The user is able to define the data source to be used, the structure of the data to be sent (if any), the target consumer of the data and (if needed) the transformation to be applied on top of the data.

Choice of Engine

Consumers of the data collected by CDPz can be multiple for the same data as well as for a selection of data. IBM decided to be very open and to make CDPz the collection point for z Systems Operational Data for clients that are both on and off platform. Users can now choose whether to send data to IBM Operations Analytics products, or to other analytics engines (e.g., Splunk, Elastic Stack, Kafka/Hadoop, Logstash, etc.). A specific integration path was created and made available in March has CDPz sending z Systems Operations Data directly to Splunk, because Splunk was ranked number 1 in worldwide ITOA software market share for 2015 by IDC, with over 85 of the top Fortune 100 customers using Splunk.

IBM recognizes the importance of giving clients the choice of which analytics engine to send their z Systems operational data. Because of the advanced filtering capability, CDPz is able to limit the amount of data sent to Splunk (or to the other engine) based on real needs, limiting the cost of the overall solution. Moreover CDPz even allows scenarios in which different data is sent to different analytic engines simultaneously, based on the planned use of the data. In this way it’s possible to realize a scenario where, for example, security z Systems Operational Data is sent to a Splunk instance, because that Splunk instance is used by the enterprise security team for real-time monitoring of the overall IT infrastructure. Simultaneously application logs are sent to an Elastic Stack because another team performs real time log monitoring on that platform.

Flexible Solution

CDPz is a young/old product: young because it was just recently made available by IBM, old because it is based on 30-plus years’ experience. It offers a great flexibility to satisfy the most advanced and complex user scenarios and needs with a simple and user-friendly configuration and setup.Main strengths of CDPz are the ability to filter on the content of the data that allows the users to carefully select what data to send where. Another strength is the ability to send data simultaneously to different targets.

Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.

comments powered by Disqus



2017 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

All Together Now

A Centralized Business Analytics Environment Delivers Greater Value

Unlocking the Secrets of Success

An introduction to business analytics

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
Mainframe News Sign Up Today! Past News Letters