“It's All In How You Do It ...”
© by Mike Robinson
I'll never forget the time when a friend of mine who was an accomplished stage magician showed me how one of his bewildering tricks worked. The experience briefly spoiled the show for me: when you know just where to look and what to look for, just a magic-show isn't entertaining anymore. But his revelation also (very briefly) sparked in me that notion that must be oh-so-familiar to stage magicians (and to the proprietors of shopping-mall magic shops) ... “gee, I could do that!”
Let's just say... that stage-magic is quite a bit harder than it looks.
The basic principles of stage magic are obvious and easily-learned, but the hallmark of a professional magician is that he or she is able to perform the tricks consistently, night after night, while keeping the attention of the entire audience pleasantly (and bewilderingly) focused upon the show itself. “The real trick of magic,” as he said, “is not ‘the trick.’ It's the whole show. ”
Software development, to a great many people, is a whole lot like stage magic. We sit at our desks and perform magical tricks, but they came to watch the show. But honestly, how many people really enjoy watching what we've produced? How often does our patron walk out of our theatre so thoroughly satisfied that they can't wait to tell their friends to alter their vacation plans? Did they see us fumble? Did we drop the poor bird? Did they come away from our pre-launch demonstration certain that they now had every single thing we'd promised them, in those initial meetings long ago, and more? Or would they merely say about us: “Foo! You are nothing but a charlatan!”
We consistently fail, as an industry. Let's start there: yes, we do.
In fact, we fail so much, and we've failed so consistently, that we've written all kinds of books and listened to all sorts of flamboyant speeches whose only real purpose seems to be to assuage our corporate consciences. “Our industry is different from all the rest,” they assure us. “There's never been anyone in the history of mankind who's done anything like what we do.” “We play by a different set of rules.”
And then there are the buzzwords, all of them (of course) quite flattering: “agile,” or for the testosterone-charged twentysomethings among us, “extreme.”
Poppycock.
I mean, it's just not so.
“How do you get to Carnegie Hall?” “Practice, man, practice!”
Okay, we've heard that old-saw a hundred times. So why do we still laugh? Because it's still true. Every so-called “overnight” success actually spent years plying their trade, practicing their stage-tricks in the privacy of their bedroom, and doing all of the things that I didn't have the perserverence to do with the very same little tricks that they also bought from the very same shopping-mall magic store. In time, with the very same little gadgets, they made magic.
“They had talent,” we might be quick to say, as though those people who were successful in their endeavors somehow possessed a sort of magic-elixir that we poor ordinary souls do not. “Well then, they had determination,” we might then assert, as though the willingness to starve must be a necessary pre-condition for a forthcoming sumptuous feast. But I would argue that achieving success in stage magic, or in musical performance, or in computer-project deployment at your place of business, derives from none of these noble qualities. The people who are truly successful in any endeavor are also consistently successful, because they planned it that way.
Every move in a stage-magic show is meticulously planned. If you don't believe me, go see it twice. Well, you might have to see it several times, because performers do inject a certain amount of variation into each show just so they don't fall-asleep from boredom during the performance, but if you see a good performance enough times in a row you'll begin to notice the scrupulously planned details, being carried-out with rigorous consistency ... and with grace and style.
“Success,” then, is no accident. Of course there will be problems now and then ... sometimes, you will drop the bird ... but those have been planned-for also. Even when a particular device simply doesn't work, or a stagehand completely misses a cue, both the on-stage performer and the entire production team have worked out a plan to make sure that the audience doesn't notice; or at least, has no reason to care. “Some of the world's best tricks have been invented spontaneously in just that way,” so he tells me.
Even in magic, luck smiles brightest on the well-prepared.
From “One,” to “Many” ...
When “The Beatles®” first started playing gigs, it was just three instruments,
a couple of amps
and a drum-kit in the back of a truck, which they loaded, unloaded and drove themselves.
On their final tour, several dozen tractor-trailer trucks and crew-buses maneuvered to each stop while
the band-members and their personal entourage traveled ahead.
If a paper linen napkin
had hit the floor while one of them was eating dinner, there were half-a-dozen people there
to pick it up.
Yet in each case the eventual performance was the same: the “Fab Four” stood on a great
big stage much like the earliest ones in Liverpool, and they sang songs that they had written
to a crowd of screaming fans.
Obviously, although the essential elements of a Beatles® performance remained the same, the scope and complexity involved in carrying out those performances grew enormously. A massive amount of planning and logistics went into each gig, all of it out-of-sight to the screaming fans. This level of planning and organization was absolutely necessary to running an enterprise of the scale that their sold-out concerts had become. Had they not been able (or willing) to scale-up the performance to such a degree, they would have merely continued to play bars in Liverpool until one of them finally got tired of driving that old truck and lifting those speakers.
I believe that one of the key reasons why software projects fail is that too many so-called “software professionals” are still schlepping-around their own instruments. In their own hearts, and in their fundamental business-practices, they are still “one-man bands.” They know how to do magic tricks, know how to write songs and sing them for their friends, but they have no idea how to scale-up that production toward the point where multiple people are involved. And maybe they really don't want to. They're used to being creative “when the inspiration strikes,” and they work until wee-hours because “that's what worked before, and it was glorious...” But when they encounter a situation where those heroics don't work, they simply don't know what to do.
And they try to bluff. They make promises, and when those promises don't work out they make more promises, and on-and-on it goes until all the roadies have left them and they suddenly realize they're just one more washed-up musician telling “worn-out stories of glory days” at a soup-kitchen.
In its simplest analysis, computer programming is all about the details. There are literally hundreds of thousands of them in a system of any size, and they must be interlocked perfectly. Any discrepancy will produce failure, because every fragment of a subsystem which affects any particular thing is thereby functionally coupled to every other such part ... whether those other parts are nearby or far away; the coupling obvious or obscure.
The talented individuals who are most-attracted to computer programming in the first place are those who are good at keeping track of details ... which they keep track of in their own heads. Most programmers describe the experience as karma ... “I can just see it.” But nobody else can, and that becomes a serious problem with a large and complex project.
In a large project, multiple people must be involved and they usually must be involved well-in-advance and for a long time. Furthermore, they must all know what the target is going to be, long before anything is actually built. This means that the programmers can no longer “make it up as they go along.” They must instead learn to apply their creative powers twice: first to develop the specifications for the project, then to write and debug the actual code. The specifications that are created must be binding, either in a literal legal sense or by an equivalent internal mandate.
In our experience, programmers almost inevitably resist such an idea, calling it “absurd” and the people who're advocating it, “clueless.” That is, until they actually try it. When that happens, there's often a role-reversal that's just as forceful as their original attitude. They wonder why it took them so long. And so do we.
Paperwork... paperwork...
The word “paperwork” inevitably calls-up really bad mental associations: your accounting teacher, for example. Applying for financial aid. Applying for anything. The term is disparagingly used for a futile exercise in imagination where you're stuck writing a bunch of made-up sentences when you could be doing “real work.” Strange as it may seem, my central premise is that (in this case, at least) paperwork is good for you.
Yes, you need to do it.
You need to do it the right way, and with the proper goals in mind.
You'll wonder why you ever thought it was possible to do things any other way.
The essential secret of most stage-magic tricks is often nothing more than a (necessarily) palm-sized little widget, made with buttons and strings and paper cards and other “spare” parts. Likewise the source-code quantity of even a major business software system might fit easily into just a few hundred closely-printed pages. But as my friend said, “ ‘the trick’ is not the trick.” Both the source-code of a major software system and the little secret widget of a stage-magic trick are, in a very real sense, just the critical means necessary to achieve an important end. They're also something the audience will never see (unless you drop the bird), and will never care about. You positively can't pull-off the trick without that gadget, nor deploy a software application without all that exactly-correct source code, but if your attention and your planning remain focused only on such gadgets then you're a trick-designer, not a magician. You're a coder, and maybe an incredibly good one, but you're not a software engineer and you're not qualified to design and lead a total engineering project. You won't be happy doing it. Go sign-up in a supporting role at somebody else's show, and make sure it's a well-run show.
A stage-magic show is a big team effort. Even with one person on stage (with “his lovely assistant”), the show itself involves the cooperative efforts of dozens of people who are wearing black shirts, black gloves, black masks and black pants. Every move is planned. The master-plan is used to derive a series of subsequent plans: lighting, prop-movement and placement, costumes, choreography, even trailer-truck loading plans. What starts out as a simple concept, such as David Copperfield's famous “we're going to make the Statue of Liberty disappear,” positively explodes into a mountain of what ill-informed people might derisively call “paperwork.”
The Play's the Thing ... Sit Back and Watch
Remember, a commercial magic act is a lot like a computer software project: eventually, “the show must go on.” The user turns on his computer, clicks an icon on the screen, and “the curtain is now up.” Every single thing that happens or doesn't-happen now is quite beyond your control. The user, like any stage audience, comes into your theatre voluntarily and with the full expectation of receiving every single thing they've paid money for. But they've also got a sackful of tomatoes, and it's a short distance to the ticket-booth to demand a refund. The show, once launched, will run until the user chooses to end it. There's absolutely nothing that you can do, at that point, to alter any outcome that may occur. The quality of your prestidigitations (“magical hand-waving”), practiced though they may be, won't make the slightest bit of difference if you drop that bird, or if you open the container and the bird's not there, or the container's not in exactly the right position or if any of a hundred million things happened that could happen but that ... don't.
So, what's really important in a software project management plan?
Follow exactly the same procedures that you might follow in any large capital project involving the work of multiple people over many months:
- Don't do things “fundamentally different” for software projects. Don't follow “methodologies” no matter how attractively they might be packaged and no matter how many success-stories accompany them. Follow your instincts and intuitions when evaluating any claim.
- Start with rigorously-defined expectations. The time to evaluate your options, and your motivations for selecting them, is before the project is underway, or at the earliest possible stages. Not every objective will be met immediately, but every objective should, insofar as possible, be anticipated and considered.
- Don't “assume” that you know how to do it, or even that a particular project should actually be done! There are many more options out there today than you might think. Don't take one person's word, no matter where that person lives on the org-chart.
- For each defined objective, consider both the things that must be done in-advance or at the same time, and the things that could block that objective and keep it from succeeding. Explicitly consider the timeline, as a system of prerequisites and co-requisites.
- Set long-term time-and-budget boundaries for the project, with intermediate administrative goal-points so that you will have plenty of advance notice if the project is falling off track. However, anticipate that the developers will only be able to provide short-term time estimates due to the inherent complexity of the task. Require them to provide daily estimates and time-logs, which must be submitted (electronically, of course) every day, and to project the amount of time required to do each task before actually doing it. Developers tend to be fairly consistent in their estimating habits.
- Manage the project's technical progress through “short hops.” The work will naturally begin to feel busier as you begin to see the end in sight, but the workflow and the general execution of that workflow should remain, as much as possible, exactly the same throughout. “Steady, steady wins the race.”
- Appoint full-time experienced project managers. Don't delegate this task to developers. Any complex capital project requires effective management, and there is no exception to this rule. Notice also that “project management” consists of much more than simply making sure that obligatory paperwork is submitted on-time. (That's administrivia, not management.)
- At the start of the project, maintain an issues log and a separate bugs database. Insist that all issues and problems must be immediately logged, but do not attach a stigma or a black-mark to the size or to the age of that list. Use software that allows you to integrate issue-reporting and problem-reporting with source-code version control and project resource-tracking. Use this as a central repository for all project-related documents.
- Invest in training. Software professionals need to grow and to be exposed to new ideas.
- Insist that the entire development team work only regular business hours. “Nine women cannot make a baby in one month.” Although developers historically tend to prefer non-standard working hours, and that might be okay, work against the notion of working more than eight hours a day, or on weekends. Software development is like a relay-race, not a sprint. You need to foster steady progress toward the assigned goals and milestones, with everyone on the team(s) working together both in the planning stages and in the execution and follow-through. They need their sleep, their break-time, their outside lives. Neglect of these things shows up very quickly in product quality. Developers sometmes need to be gently reminded that “the work will still be there tomorrow.” This kind of perspective isn't just for popularity-points ... it's smart business for you and your company.