Understanding the Planning Process
This is Chapter 2: Understanding the Planning Process from the book Planning Smarter: Creating Blueprint-Quality Software Specifications (ISBN: 0-13-065414-0), written by Tyson Gill, published by Prentice Hall Inc..
© Prentice Hall Inc. All rights reserved.
Chapter 2—Understanding the Planning Process
We've got big trouble in Software City. In the famous musical The Music Man, a fast-talking salesman comes to a small town named River City, Iowa. He tells the gullible folks that they "got big trouble." He warns them that the new pool hall will induce their children to fall prey to bad habits. The only thing that will save them, he declares, is to provide a wholesome alternative, such as a boys' marching band. Of course, it just so happens that the salesman deals in a fine line of quality band uniforms and instruments.
Like The Music Man, I'm here to tell you that we got big trouble in Software City. But unlike The Music Man, I'm not saying that just to peddle a textbook equivalent of 76 trombones and 110 cornets. While some authors applaud the state of the industry, I have serious concerns. While there are particular teams that do experience success on certain projects, such successes are not the norm.
Sources such as objective statistics, self-grading, client reaction, and anecdotal experience all confirm that the rate of failed software development projects is embarrassingly high. Failure rates from 80 to 90 percent are commonly cited and generally accepted. Of course, the accuracy of these figures depends on how success or failure is measured, but by any measure, the situation is not good. The need to improve our software development processes is very real.
To make the picture worse, the cited failure rates reflect only projects that overtly failed to meet objectives. They do not include "successful" projects that cost far more than they might have, or took far longer to complete than was necessary. I disagree with authors who claim that failure rates are artificially high because they are based on unrealistic expectations (DeMarco, 1995). On the contrary, the expectations for minimal success across the industry are far too low. Furthermore, success is most often measured by short-term deliverables, not by long-term maintainability. If failure to maintain were also considered, the picture would look even more dismal.
Poor performance is a pervasive and recurring problem that affects the credibility, effectiveness, and long-term survival of the software industry. Despite an undeniable need, fewer potential clients are willing to risk precious capital on custom development projects. They have either heard of, or they have experienced first hand, the development nightmares that all too commonly occur in these projects. As of this writing, the market for custom application development continues to diminish. I don't believe this is due to a lack of need, but rather to a lack of faith in our industry to deliver.
Clients are not the only ones who are disenchanted. Developers are perhaps even more frustrated. Talented developers don't want to waste their talent over and over again on doomed projects. New developers get disillusioned with the business and find other work. Managers burn out from having to fight the same client relationship fires all day long. Owners of development shops sometimes just give up application development and change their business to provide more profitable and less problematic software services.
We face the real possibility that the fledgling software industry will collapse into a much smaller number of players producing mass-market solutions. It may never mature into the lucrative and diverse industry that we would all like it to become.
This should not, and need not, be the future of our industry.1 Custom software development can and should be a thriving, rewarding, and cost-effective industry for both the consumer and for the provider. How is it that an industry that specializes in organizing and streamlining other business practices can itself be so abysmally unorganized and inefficient?
When a problem is identified, our first reaction is to look around for someone or something to blame. Where does the blame lie here? Do we blame the people involved for creating this situation? Have they simply failed to implement the excellent processes and practices that are well-documented in the literature? Many would say yes. However, intelligent, talented, and experienced professionals staff many failed projects. We cannot simply dismiss them all as incompetent. We should search for root causes of the problems, evolve effective countermeasures, and apply them. We should create a continuous process improvement ethic.
Are the "best practices" that we turn to for guidance perhaps not as good as proponents would like us to believe? I am not willing to say that. As I said before, the practices and approaches recommended in the literature are mostly sound, practical, and based on proven experience. We can't minimize their value and effectiveness.
So what is it then? What is out of kilter in the software development industry? In my opinion, the source of the problem is not individuals or any specific practices, but overcomplication. In our conscientious efforts to minimize risk, adopt best practices, meet budget constraints, prevent wasted effort and so on, we introduce overly restrictive and complicated schemes which do not meet the fundamental requirements for productive planning. Our solutions, in part, make worse the very problems we are trying to avoid. We expend far too much time and effort in project planning and management with too little payback.
The key to improved success is to simplify the process. Eliminate unnecessary effort. Expose and eradicate the sources of those problems that force the application of overly complex and counterproductive practices. This is best accomplished by focusing on the production of a simple but effective software blueprint during the project-planning phase.