Understanding the Planning Process
2.5 Excellence Is Attainable
After the grim portrait of software performance that has been painted so far, you might be tempted to resign yourself to the status quo. You might rationalize that although you have problems, so does everyone else. You might conclude that you can't do any better. You might further conclude that the quality and productivity of the software industry are as good as they can get and it is the expectations that are out of line.
"The work of software development is largely incompressible." (DeMarco, 1995)15
However, there are projects that do succeed significantly better than others. They prove that you can do substantially better than the norm. Perfection may never be attainable, but excellence certainly is.
"Large information systems don't have to take so long, they don't have to cost so much, and they don't have to fail." (Highsmith, 2000)16
This wide gap in performance may be taken as grounds for pessimism, but it also offers hope and encouragement. It proves that there is tremendous room for improvement and that those improvements are achievable.
Does this mean that there is a 600-to-1 difference in ability and talent among the individuals involved in these projects? Not likely. Even more unlikely is the fact that there would have to be a 600-to-1 difference in the average talent of the various teams, making the actual individual range of individual differences far greater.
Nor are the successes merely a matter of luck. Yes, you can get stuck with a nightmare project with little hope of success. And there really are those rare dream projects that we all fantasize about. But no matter how the cards are dealt, successes are still made. There are companies that can reproduce success. They can win almost every hand. They overcome undesirable factors and capitalize on the favorable ones. These companies are not the Wizard of Oz. There are real programmers behind that curtain, like you, Tin Man, and you Scarecrow, and even you, Cowardly Lion. But they do have something you probably don't have—an effective planning process.
If you don't yet have your planning process down to a finely honed craft, you may want to consider the value of a structured software blueprint as the critical deliverable of your planning process. There are many other important components also. Configuration management, testing, code reviews, and coding standards are just a few. But the blueprint is the most critical and the most commonly overlooked.
2.6 A Software Blueprint
The goal of any software planning effort should be to analyze the business requirements and translate that understanding into a specification that facilitates the development process as efficiently as possible.
If you don't know where you're going, it is difficult to get there and impossible to know when you have arrived. Most planning efforts proceed by fanning out and mapping everything as they go, hoping that the end result will include a route to the ultimate destination. They craft their planning deliverables in ad hoc fashion as they go. The functional specification ends up becoming an apparently random maze designed to bewilder and befuddle the developers.
This problem is inevitable when you don't have a clear planning goal defined from the onset. The software blueprint defines a tangible goal for the planning process in the same way that a Form 1040 defines a tangible goal for the tax preparation process. It should provide developers with exactly the type of information they need to construct a program, in the same way that a house blueprint provides the construction crew with exactly the information they need to build a house.
While analogies and comparisons are never perfect, they do help make ideas more familiar and meaningful. House building is probably the most commonly cited analogy for software development. There certainly are a lot of similarities, but there are a lot of differences as well.
Software development is a very customized process. However, house building can also be highly customized. The elements that make up a house are fairly simple. Floors, walls, doors, and windows make up every house. Likewise, there is a fairly simple set of pieces that make up every software project. If we can arrange these pieces into a standard blueprint that organizes them, then that blueprint should be all we need to describe most software projects effectively for the builder's purposes.
A software blueprint is a single, simple, and standardized format to specify software requirements. Builders couldn't be expected to construct a new house without a blueprint, yet software developers are routinely asked to build software without an equivalent blueprint to work from.
Most software specifications do not even begin to serve as an adequate blueprint. They are designed to serve many purposes, but not specifically to meet the needs of the developers. They do not provide developers with the exact information they require, at the necessary level of detail and consistency, and in an easily accessible format.
If software were a house, here is how the usual planning and construction process might proceed:
- The salesperson initiates contact with the buyer and tells the project manager what the buyers are prepared to spend.
- The project manager has several meetings with the buyers. He gets a general idea of what they are looking for and prepares a Vision Document documenting the problems with their current house and specifying what size house the clients need. The Vision Document cites the projected cost (coincidently just a bit more than the buyers said they were willing to spend) and describes how much they will gain compared to their current home.
- The homeowners think that sounds pretty good, so they meet with the house designer every day for the better part of a year discussing what kind of colors they like, what kind of dimmer switches annoy them, and the hours of the day when they use the bathroom and kitchen.
- The house designer collects all the information, including tax returns, magazine pictures from Better Homes and Gardens, and poetry composed by the homeowners to describe their vision of the new house.
- The house designer prepares some usage scenarios which summarize the times that the homeowners are likely to use the garage door opener as well as some transition diagrams illustrating in detail their expected traffic patterns in the new house. He bundles these up with the magazine pictures provided by the prospective owners, and hands all this to the construction supervisor.
- The construction supervisor, already told what the house will consist of, what the buyers will pay, and when they need it completed, goes about the task of fabricating a construction plan and cost breakdown which reflect the timeline and total cost already predefined. The detailed estimate, tailored to meet the predetermined cost, reassures the steering team that their initial estimates were right on.
- The construction supervisor goes about gathering a construction team whose members laugh or cry, according to their particular nature, when they are told what the house will include, what it will cost, and when they are expected to complete construction.
- The construction supervisor hands off the house plan to the construction crew. In due course, the crew sets about to somehow build a "bedroom with two doors" and a kitchen that "radiates warmth" as they understand it from the documents they were given.
- The management team, distracted with planning future houses, doesn't pay much attention to the builders for a few months. The homeowners are too busy at their current jobs to offer much information. After all, they already took too much time off for all that earlier planning.
- The builders aren't sure where those two doors in the bedroom should be. They ask the planner and he tells them that the information is right there in the owner's childhood diary. The builders give up and put one into the hall and one into the bath.
- Finally when the house is nearly complete, the homeowners make a walkthrough. They list about 179 first pass changes, citing the fact that it clearly says in the wife's diary that she always wanted a door to the patio from her bedroom. Oh, by the way, where IS the patio that was inferred by that passage?
- The construction supervisor is sacked for not including the patio, and the crew, already far over budget and months late, works far into the night to make the changes.
That is pretty much the typical scenario in the software business. Needless to say, no home construction company could survive very long if it operated this way. In reality, the life expectancy for software firms is not high.
Page 7 of 8