MAINFRAME > Business Strategy > BI and Analytics

Apache Spark Is a Superior In-Memory Analytics Engine

Apache Spark

Q: What's the best in-memory analytics engine? Why?

Apache Spark is one of the best in-memory analytics engines out there. It’s gained a lot of momentum in the big data and analytics industry, and I expect that momentum to continue to accelerate for several reasons.

For one thing, Apache Spark is a fast in-memory analytics engine with a complete, functional runtime library. Its performance is scalable in terms of doing in-memory analytics. It provides business value by helping organizations process data more efficiently, and it helps keep pace with the influx of data. In addition, Apache Spark reduces the time it takes to convert raw data to business insights, offering a strong competitive advantage to enterprises that use it.

The fact that Apache Spark supports the four popular languages of big data—Java*, Scala, Python and R—makes it very attractive for developers. It increases developer productivity because it provides very consumable APIs.

With Apache Spark, you can use the same APIs to process streaming data as well as historical batch data, so it provides a unified development environment for data scientists, statisticians and application developers who want to analyze data.

Analytics applications, as we know them, are very memory intensive, but Apache Spark can provide considerable performance and scalability benefits to a greater extent than competing in-memory analytics engines.

The last point I’d add is that IBM included Apache Spark as a main component of its IBM Open Data Analytics for z/OS* offering for good reason. With Apache Spark, along with its other two primary components—Anaconda and Python—IBM Open Data Analytics for z/OS can analyze data in place, gain additional business insight and preserve the security of where data originates.


Elpida Tzortzatos is a senior technical staff member and team leader for the z/OS Core Development Team.

comments powered by Disqus



2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.


AI-Driven Technological Progress

All Together Now

A Centralized Business Analytics Environment Delivers Greater Value

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
Mainframe News Sign Up Today! Past News Letters