Celebrating 10 More Years of RPG Progress
IBM i experts Jon Paris and Susan Gantner reflect on a decade of progress for the RPG development language.
We're celebrating the arrival of a new decade* by looking back at some of the IBM i Application Development highlights over the past 10 years.
*(We understand that technically the new decade doesn't begin until 2021, but most everybody seems to think it starts now and it feels like a good time to look back…)
In this, our first article of 2020, we'll be giving an overview of what we personally perceive as the most significant changes in the world of RPG.
Not Bad For A "Dead" Language
For many years now, we've been hearing that "RPG is dead" and "IBM isn't enhancing it anymore", and on and on. The first time we heard the "dead" comment put forward with any conviction was back in the 1980s and it has been recurring with monotonous regularity ever since.
Obviously, RPG is far from dead. In fact, the last decade has seen some pretty remarkable growth in the capabilities of the language and put the lie to thoughts of IBM abandoning it.
During the past decade we've seen RPG opened up to allow ISV and user-written code to integrate seamlessly without impacting the simplicity of the language structure. Powerful and flexible features can be added as if they were part of the language, enabled by compiler-provided hooks. We've also seen flexibility and freedom enhanced not only syntactically with free-form declarations but also with more flexibility in many aspects of RPG coding—from array sizing to procedure exit behavior and using long SQL column names in your RPG.
"Opening Up" RPG
Perhaps one of the most significant changes, in part because it set the scene for things to come, was the introduction in 2010 of Open Access (OA)—or as IBM insisted on naming it "Rational Open Access: RPG Edition." (We're still waiting for a COBOL version - but not holding our breath!)
In our view, the most significant aspect of OA was not that it enabled ISVs to provide a better approach for taking green-screen applications to the web. Certainly, that had a huge impact and companies such as Profound Logic, Fresche, and Rocket—to name just three—were quick to exploit its capabilities. No, the real power of OA is revealed in the phrase "it enabled ISVs." OA was the first step in allowing third parties (including in-house developers) to develop extensions to RPG that were fully integrated into the language's structure. OA was followed by DATA-INTO, and more recently DATA-GEN, but we're getting ahead of ourselves and we'll return to them later.
Sadly, while many ISVs embraced OA for browser enablement, we were surprised by the lack of initiative shown by ISVs in other areas. To the best of our knowledge, only RJS (now HelpSystems) has introduced an OA-based RPG extension. It is part of its RPG2SQL Integratorsuite and utilizes OA to allow an RPG program to interact with other databases such as Oracle and SQL Server through the use of conventional RPG op-codes.
While we know that a large number of shops have adapted the OA utilities that we have published for handling CSV files, calling web services and other things, it’s still disappointing that so few ISVs or open source groups have latched on to the potential that OA presents.
Following the introduction of OA, things were a little quieter for a while and IBM focused on "tidying up" a few things. For example, a number of additional options were added to XML-INTO (which was originally introduced in 2006) that filled gaps that had prevented it from becoming a fully-fledged solution for handling XML documents.
And then in 2013 (was it really that long ago?) came the announcement that the old H, F and D specs were to be replaced by new free-form options. As those of you who have read our series on The History of RPG IV will know, the original plans for the language called for it to be free-form. That idea was dropped in favor of the hybrid version that was introduced as RPG IV back in the V3R1 time frame.
We had always hoped that one day our free-form dreams would be fully realized. In 2013, free-form coding was still restricted to columns 8-80 but at least everything was free-form. We wrote at the time how much easier this made it to introduce new programmers to the joys of RPG. An opinion that has been strengthened with each group of newbies that we have trained in the last few years. Of course, PDM/SEU doesn't support the free-form declarations, but since we would never attempt to teach new RPGers such an antiquated tool anyway that has not been an issue for us.
Of course, it wasn't until 2015 and the introduction of **Free that RPG finally became free of all* columnar restrictions. Now not only can you start your code in column 1, but there is no longer any limit on the line length. Oh happy day! (* Note: It is somewhat ironic of course that the onlyfixed-form aspect of RPG today is the positioning of the **Free directive, which must appear in the first 6 characters of the first line of the source member!)
More Flexibility and Readability
Once free-form declarations were available it just made sense to allow RPG to use SQL's longer field names, or indeed those provided by DDS ALIAS names. That support, along with enhancements to Data Structure (DS) Input/Output was added in 2014. Not only could the longer names be used when generating externally described DS, but the ALIAS keyword could also be added to file declarations allowing externally described files to use the longer field names.
2016 saw the arrival of the ON-EXIT option for subprocedures. This provided a convenient mechanism for handling any errors that might occur during the execution of a subprocedure as the code in the ON-EXIT section is always executed, whether the subprocedure is returning normally or has crashed and burned.
Then in 2017, the new free-form data declarations were enhanced with the ability to directly nest data structures one within another without the need to use LikeDS. This makes it so much easier to code DS for operations such an XML-INTO because you can literally code the DS to have the "shape" of the XML it represents. We were particularly happy about this, not just because it was useful, but it was also further proof that IBM does indeed listen when we write up Requests for Enhancements (RFEs). We know that because Jon wrote the RFE requesting this particular enhancement.
Opening Up With DATA-INTO
Speaking of RFEs: Ever since the introduction of XML-INTO RPGers had been asking for an equivalent for JSON, which was rapidly growing in popularity. Rather than provide JSON-INTO though, IBM continued the line of thinking behind the design of OA and instead in 2018 added DATA-INTO to the language.
DATA-INTO works in much the same way as XML-INTO (i.e., it takes a document and unloads its content into a DS) but with one big difference. Instead of IBM supplying the parser, as they did with XML-INTO, DATA-INTO allows us to supply the parser. This has the significant advantage that when the next "flavor of the month" for data interchange comes on the scene, we won't have to wait for IBM to come up with a new op-code. We’ll just need a new parser, and that in turn means that you won't have to be on the latest release to be able to use the new facility. You can learn more about using DATA-INTO here.
Opening up with DATA-GEN
In addition to asking for a JSON version of XML-INTO, RPGers had been asking IBM for what they usually described as a "reverse XML-INTO". i.e. an op-code that would take a DS and produce an XML formatted document from the content. Once again, we were lucky in that IBM took the path established by OA and DATA-INTO and met this request by enabling us to supply our own formatters.
This facility was introduced in 2019 with the op-code DATA-GEN. Under the covers, RPG calls the supplied formatter, advising it of the value, name and data type of each element that it retrieves from the DS. From the information passed, the formatter adds whatever tags etc. that the data format requires and passes it back to RPG. RPG then merges the results into the file or variable specified. As with DATA-INTO this ensures that no matter what future data transfer mechanism comes along, RPG will be able to handle it. You can learn more about using DATA-GEN here.
Object Oriented RPG?
Despite many requests for it, IBM stated long ago that they had no intention of ever making RPG an object oriented (OO) language. On the whole we were happy with this situation. Nevertheless, there were features normally associated with OO languages that many of us felt would be a useful addition to RPG.
In 2019, the first such capability was added to RPG in the form of procedure overloading. If you are not familiar with this concept, the simplest way to explain it is to say that it enables you to use the same procedure name for a range of routines, each of which takes a different parameter list. The alternative, and the only option we have had to-date, was to use slightly different names for each of the variants of the procedure. If you want to know more about this capability, we wrote about it here.
No Crystal Ball Needed: Variable Length Arrays
Until now, every time you defined an array in an RPG program, you had to decide in advance the maximum number of entries that it would ever need to hold. This required that you create a balance between memory usage and safety. We found over and over that, while no matter how hard we tried to get the number right, at some time down the road there would be a program crash because of an unexpected increase in the number of array entries needed.
In 2019, this problem was resolved with the introduction of variable length arrays. This is probably the longest outstanding requirement for the language. We can remember being asked when this would be available way back in the V3R1 time frame when RPG IV was first introduced. The only downside was that this was one of those rare enhancements that was so pervasive in its impact on all aspects of the compiler, that it was only made available for V7.4. As a result, adoption will be a lot slower than it has been for most of the other features added to the language in recent years.
If you'd like to know more about this feature, you can read all about it here.
And the Next 10 Years?
Just as 10 years ago we had no idea that some of the enhancements we have seen in this decade would ever be part of the compiler, so we have no idea what comes next. In all probability, some of the features added will be to handle situations that today are on the fringes of commercial programming tasks or that have yet to make an appearance outside of research labs.
Another thing that IBM has to take into consideration is that sometimes it may be better to add further functionality to Db2 than to RPG. After all, if it is available in Db2 then it can be used from RPG and COBOL and ...
One thing is for sure. RPG will continue to grow and remain at the center of our IBM i applications. And you can help determine its future direction!
How? Well there are a number of RPG RFEs out asking for various new features, but which of them will make it into the compiler in the next 10 years is anybody's guess. Youcan have an impact on which are implemented by voting for the RFEs that you would most like to see. You can find the current list of requests here.
Don't forget that if you don't see a feature that you think should be added to the language then write up your own RFE! The new RPG Cafe provides details of exactly how to do it.
For ourselves the number one thing that we'd like to see would, hopefully, not be restricted to just RPG. What's that you ask? Better and simpler interoperability between the new open-source languages and traditional ILE languages. We're not sure what form it should take but we know it is an area of focus for IBM and hopefully enhancements will begin to emerge in the not too distant future.