AIX > Tips & Techniques > Application Development

Demystifying the Myth of Measuring Programmers Via Metrics

This is the second in a series of articles in my “how-not-to” series based on my more than 35 years of work in mainframe IT, reviewing mistakes I’ve seen or participated in and helping others to avoid them. My first article, “End Users: A Programmer’s Best Friend and Worst Enemy,” can be found here.

Maybe it’s because many IT departments have traditionally reported to accounting and finance; maybe it’s because most people don’t grasp the nature of computer (especially mainframe) development and support; maybe it’s because IT staff are paid fairly well due to job complexity, training and experience requirements; maybe it’s mistrust or jealousy or fear, but I’ve run across several situations where upper management has insisted IT find quantitative ways to measure performance. That might be desirable. It’s possible for some jobs but is untenable for system and application programming and support.

Programming Doesn’t Just Mean Writing Code

To the uninformed, programming means writing code. Merriam-Webster’s defines computer programming as “the act or job of creating computer programs.” The Oxford dictionary defines it as “the action or process of writing computer programs.” If only it were that simple! From my experience, being a computer programmer involves a few more things:

  • Testing. It’s one thing to write a computer program, another to determine the program works correctly. To verify that, various scenarios need to be tested, using different sets of data, producing correct output based on various inputs and validating different circumstances produce desired results. Quality assurance units do more advanced testing, but the programmer is usually responsible for initial, basic testing.
  • Debugging. When testing reveals a program malfunction, the problem must be identified and fixed. That isn’t always straightforward, and sometimes the problem lies elsewhere in the application or system.
  • Defining requirements. While the end-user is primarily responsible for determining requirements, programmers often participate in feasibility analysis and integration.
  • Documentation. Documenting programs and applications is the most undervalued, ignored aspect of programming. I’ve worked in a shop where documentation–flowcharts, program comments, etc.–was strongly discouraged under the assumption it slowed coding. Maybe so for initial coding, but come back a few years later to debug or enhance the code; it can be a nightmare. Even worse, ask someone new or unacquainted with the code to do it. Documentation is indispensible.
  • Screen design and definition. While not programs, screen definitions must be coded or generated for online applications.
  • Data design, definition and population. Programmers are usually involved in structuring files or databases, creating copy books and loading, although it may be in a mostly incidental role.
  • Project management. Additional programmer time is consumed in time reporting, status reports, etc.
  • Languages and development platforms. It takes more lines of source code to write an Assembler program than COBOL, and it takes more lines of COBOL in a native program than when using a program generator, let alone Object Oriented languages like Java or C#.

To net it out: Most average programmer time is spent in non-coding activities, activities not quantifiable. And when it comes to measuring lines of code, it can be an exercise of comparing apples and oranges.

Code Quality

Another aspect of coding missed when measuring a programmer’s lines of code is:

  • What’s the code quality?
  • Is it well written and easily maintainable?
  • Does it take advantage of programming methods such as structured programming and object-oriented programming?
  • Have performance-related considerations and methods been employed?
  • Has the program been flowcharted and otherwise laid out to streamline and optimize?

These aspects aren’t measured or quantified with a line count metric. Indeed, these considerations tend to minimize lines of code and are at odds with a line count metric. Producing maximum line count in the least time is almost always contrary to code quality.

Another Form Of Metric Measurement

I’ve also seen situations involving other metric measurement systems. One situation measured programmers in terms of person-hours on a project compared to estimated effort, in conjunction with number of errors involved in a move to production. This had the following issues:

  • Coding an application in the least time, and engineering a clean move to production is the contradiction of speed and accuracy–objectives at odds with each other. To accomplish one objective is to sacrifice the other when measured independently. A blended measurement is better but still contradictory.
  • In this situation, meeting project time estimates lacked accountability. In most cases the person (usually management) who estimated project effort and scope was different from the person(s) who did the work. So if the project was late or over hours, who was to blame? The estimator or executor? Why blame the programmer if the estimator was wrong?
  • Even when a programmer could provide input on the estimate, no time was allocated for research or analysis, merely an opinion. Often, the programmer had little experience with the code where the new function or application was to be put. To provide useful input, a programmer needs time to look things over, time well spent when the work begins.
  • In addition, a programmer was pressured to validate the existing estimate, especially vexing when no time was allowed for a critique.
  • Estimates are educated guesses. Pure and simple. Nothing more. Granted, there are repetitive processes, measureable in square footage, number of widgets, assembly lines, where estimates have substantial precision, but IT processes aren’t one of them. They are imprecise, filled with unknowns. In my time with IBM and Best Customer Solutions, through painful experience, I evolved the rule-of-thumb “drill deep and add 50 percent.” It worked fairly well–yet I never took a fixed-price contract: Too many unknowns.

So to measure a programmer’s performance based on contradictory objectives, educated guesses, and someone else’s performance doesn’t make sense. Obviously, this measurement system was not very popular. It wasn’t fair, it wasn’t effective, it wasn’t accurate and it wasn’t any good.


Programming is a creative process, and the programs required for a project vary widely, as do job requirements in differing environments. It is rife with unknowns, varying levels of talent and experience and imprecise and uncertain estimates. So how to effectively measure with metrics? You don’t. Programmer assessment needs to be made by managers who understand the job, who are close enough to projects to ascertain a programmer’s effectiveness, who can factor intangibles as well as tangibles.

Jim Schesvold can be reached at

Like what you just read? To receive technical tips and articles directly in your inbox twice per month, sign up for the EXTRA e-newsletter here.

comments powered by Disqus



2019 Solutions Edition

A Comprehensive Online Buyer's Guide to Solutions, Services and Education.

IBM Systems Magazine Subscribe Box Read Now Link Subscribe Now Link iPad App Google Play Store
IBMi News Sign Up Today! Past News Letters