MAINFRAME > Hot Topics

IBM Machine Learning for z/OS Is Supported By Apache Spark

IBM Machine Learning

Hot Topics content is written by IBM technical experts - the people that design and write the code. The articles provide insight into products and functionality from a developer's perspective.

Businesses must constantly adapt to changing conditions: competitors introduce new offerings, consumer habits evolve, the economic and political environment change, etc. While this isn’t new, the velocity at which business conditions change is accelerating. This pace of change places a new burden on technology solutions developed for a business.

Over the years, application developers moved from V-shaped projects (with multiyear turnaround) to agile development methodologies (with turnaround in months or quicker). This has enabled businesses to adapt their application and services much more rapidly. Examples include:

  • A saleS forecasting system for a retailer: The forecast must take into account today’s market trends, not just those from last month. And, for real-time personalization, it must account for what happened as recently as one hour ago.
  • A product recommendation system for a stockbroker: They must leverage current interests, trends and movements, not just last month’s
  • A personalized healthcare system: Offerings must be tailored to an individual’s unique circumstance. Healthcare devices, connected using the Internet of Things (IoT), can collect data on human-machine interaction and behavior.

These scenarios, and others like them, create a unique opportunity for machine learning, which was designed to address the fluid nature of these problems.

Machine learning moves application development from programming to training: Instead of writing new code, the application developer trains the same application with new data. This is a fundamental shift in application development because new, updated applications can be obtained automatically on a weekly, if not daily, basis. This shift is at the core of the cognitive era in IT.

Machine learning enables the automated production of actionable insights where the data is (i.e., where business value is greatest). It’s possible to build machine learning systems that learn from each user interaction, or from new data collected by an IoT device. These systems then produce output that takes into account the latest available data. This wouldn’t be possible with traditional IT development, even if agile methodologies were used.

Gather Information

While most clients understand machine learning, too few are turning this into action. They are either slowed down by concerns over their data assets or they attempt it one time and then curtail efforts, claiming the results weren’t worth continuing with machine learning. These are common concerns and considerations, but they should be recognized as items that are easily surmounted, with the right approach.

A common trap is to believe that data is all that’s needed for a successful machine learning project. Machine learning projects that start with a large amount of data, but lack a clear business goal or outcome, are likely to fail. Projects that start with little or no data, yet have a clear and measurable business goal, are more likely to succeed. The business goal should dictate the collection of relevant data and also guide the development of machine learning models. This approach provides a mechanism for assessing the effectiveness of machine learning models. The second trap in a machine-learning project is to view it as a one-time event. Machine learning, by definition, is a continuous process and projects must be operated with that consideration.

Machine learning projects are often run as follows:

  1. They start with data and a new business goal
  2. Data is prepared, because it wasn’t collected with the new business goal in mind
  3. Once prepared, machine learning algorithms are run on the data to produce a model
  4. The model is evaluated on new, unforeseen data to see whether it captured something sensible. If it does, it’s deployed in a production environment where it’s used to make predictions on new data.

Jean-Francois Puget is an IBM Distinguished Engineer and IBM Machine Learning technical lead for public and private cloud divisions.

Mythili Venkatakrishnan is an IBM Senior Technical Staff Member and is the z Systems Architecture and Technology Lead for analytics.

comments powered by Disqus



2017 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
Mainframe News Sign Up Today! Past News Letters