A Wheel Within A Wheel: The Conundrum of IT Project Management

An obscure E-book author, Vincent North, once wrote in his book Managing the Mechanism (still available on Amazon and iBooks) that software development was very difficult because "software is an autonomous, self-directing machine." But now I'd like to explore that concept a little further: "a wheel within a wheel."

Of course, the Biblical account in the first chapter of the Book of Ezekiel is one of the most enigmatic scenes in the entire compilation. But for the purposes of this essay I simply want to pursue the idea of: "within."

As Mr. North aptly stated, "we really are constructing a machine." But, before constructing it – no, while constructing it – we are also designing it. We're designing it for actual people who will actually be using it every day, to perform and/or to assist real-world human tasks which we as software designers can only partially understand. In creating our designs for such people, we readily see that they, too only partially understand us. There is always, as an electrical engineer might say, "an impedance gap."

I think it's useful, nonetheless, to try to address this enormous problem by separating it into "nested wheels." So, from the outside in, here goes ...

Wheel #1: "The Client"

Obviously, we are writing software "for somebody." We're trying to create a tool for them that will actually be useful in their hands. But human beings are "intelligent and adaptable," whereas computer software most-decidedly is not. Therefore, even from the outset, we are obliged to frame the very definition of our objective in terms of the digital machine – while interviewing subject-matter experts who very simply "don't think that way." We therefore easily run into two problems: misunderstanding, and omission:

• Misunderstanding: We either didn't understand what the client needed, or we framed that understanding too narrowly.
• Omission: We failed to include an important point – perhaps because the client didn't think to bring it up.

Obviously, this stage is the most-obvious point where a project can find itself "poleaxed before it is even begun." So, it gets a lot of attention in every "software project-management manifesto," usually as so-called "user stories." But even this metaphor leaves a lot to be desired, since business computer software is tasked to implement a business process, not merely a user experience.

Wheel #2: Implementation – Existing Systems

As Mr. North's book also observed, computer software consists of a vastly inter-connected system of machinery, in which any one of many-thousands of "moving parts" can impact – literally – any other part. Thus, the software design process quickly runs into the next "impedance mismatch." Namely: the successful integration of new functionality into a vast existing system.

The impact of this term must be clearly understood: "existing systems!"

Believe it or not, in less than fifty years we have gone from a world of punched-card tabulators to the computer or phone that you are right now using to read this. We can easily "focus on the future," but our clients and employers cannot. They invested in every "new" technology as soon as it became available, and, as a consequence, they now owe a very considerable debt to the technological past. The "RPG" language, for example, literally started as a software implementation of a punched-card tabulator that was "programmed" with plugboards. Millions of RPG programs continue to run important businesses today, even as the programmers who still understand them are retiring or dying off. Countless other examples exist, many of them (such as "dBase®") being products of the earliest days of "the microcomputer revolution." All of these systems continue to remain in service. Present and future designs are forced to adapt to them.

Wheel #3: Existing-Team (Technical and Social) Dynamics

The purely-human aspects of software development are often overlooked. The industry is still dominated by "lone wolves" who singularly possess the ability to write complex computer code that is somehow able to satisfy the immediate and ever-changing requirements of an actual business.

"Few and far between" are the managers who are actually able to promote true teamwork under such circumstances. By far the easiest course of action is to once-a-week inquire as to what each of the "wolves" are now doing, and to end the call by telling them to keep doing it.

Nonetheless, real-world software projects do reach the point where "lone wolves" can no longer do the job. Now it becomes necessary to separate, and then delegate, the two responsibilities that "the lone wolf" had been doing on his own:

1. Determining what new features need to be added to the [externally-visible ...] system.
2. Understanding the impact of all of this to "the software that is already there."
3. Implementing the change itself.
4. Verifying that the change actually worked, and that it didn't break something else.

It is precisely at this point in a project where Sundial Services has most-commonly become engaged, because it is at this point that the "lone wolves" most commonly jump over the fence and disappear – purposely taking their "institutional knowledge" with them.

There are a variety of ways in which a business can choose to construct a [brand-new ...] team to replace these "wolves," but their essential objectives correspond to the four bullet-points listed above. In short, the decision-making processes must first be externalized, in order that they may then be meaningfully shared.

Wheel #4: "Context-Free" Specification, Explicit "Deployment" Strategy

With this "wheel," we're finally beginning to get down to the brass tacks. We've sorted out our political differences, waved goodbye to our wolves, and now we're focused on the mechanics of changes. (But, not yet at the "source-file line" level ...)

Given that (as, once again, Mr. North observed ...) "computer software is a mechanism where every single thing is connected to every other," the process of making "any change at all" to an already-deployed software system must be "extremely formalized." And here it is that we make our most-significant departure from the customary practices of the "lone wolves." Instead of simply "deciding on-the-fly what to do," we rigorously pre-define each change before we do it. And, we pre-define how we intend to verify each change, "having done it, but before deploying it."

Every team that we have, over these very many years, introduced into these concepts have come to appreciate us. Because it turns out that all of them, having reached a certain point in "their understanding," were "ready to 'dive in.'" As it turns out: "prematurely!" Because there always was something very-important that they hadn't looked for before they started.

Wheel #5: "Third-Party Software" and User-Side Complexity

Like it or not, in every system that is developed today is entirely dependent upon mountains(!) of software that is "beyond their control." These systems obviously exist on both sides of the "host/user divide" that now rules most deployed systems. The "Wagtail" (Python-language) system that drives this web-site is constantly being updated. The "JQuery" JavaScript library now consists of several megabytes of compressed(!) software, and it, too, is constantly being updated. All of this amounts to an external influence which directly impacts any deployed system while being entirely under the control of a separate team of programmers.

These same "unwanted influences" can even be felt in the programming languages themselves. Most systems today are written, even on the host side, in "source-code form," which means that they are re-interpreted every time they run. Unfortunately, sometimes those languages change – even, fundamentally. The "PHP-7" language is very considerably different from its predecessor, "PHP-5." "Ruby" did the same. The "Perl" programming language briefly attempted to re-define itself entirely. Unfortunately, these types of changes can completely devastate an already-installed system, completely without warning. All of the sudden, the programming language no longer "means" what it used to. Features that you had taken for granted are suddenly "deprecated" by an independent team of programmers that you never signed a contract with. You literally have no recourse.

Summary: "Whew! Too Many Wheels!"

In a very, very real sense, "IT Project Management" especially consists of "managing expectations." The technology that we are presented with today is certainly not the technology of yesterday – nor will it be the technology of tomorrow. But the business requirements remain unchanged: "spend my money as wisely as you can," and "don't let me be surprised in front of my customers."

As a project manager, you are "in the middle." You must simultaneously juggle the "upstairs" expectations of your clients or employers, and the "downstairs" expectations of those who will actually perform – and, verify – the work. The single most important requirement therefore simply becomes: "communication." Your constant challenge is to simultaneously be the bridge between these two gaps:

1. Business stakeholders who utterly rely on the work-product without understanding (or, caring) how it is produced.
2. Software engineers who must be free to "single-focus" upon "their exacting task of the moment," even as they try to parse-out their daily duties among themselves.

And your common goal is, of course, to minimize business risk. (Given that those risks are "extreme.")

"Communication ..."

Yes, "all of project management really does come down to this one word." Although "business stakeholders" and "line-programmers" must approach "the common goal" in an entirely different way, it is a "common" goal. And you, as a successful project manager, inevitably find yourself frequently in the role of Janus, the Roman "two-headed god."

"The key to success," if there actually is one, might be summed-up as this: "never let either of them 'be surprised.'"