Camps are forming. The topic is the overall process for developing systems. As in any religious war, both sides know the correctness of their cause. Among the hostages are the successful use of component based development and the livelihoods of the advocates of the Software Engineering Institute's (SEI) Capability Maturity Model (CMM).
In one camp are those who believe in SEI's approach. This is called a "defined" approach. The core of this approach is the belief that software development is a process that can be progressively defined through continuous process improvement. It is the responsibility of organizations to improve their processes by climbing the SEI Capability Maturity Model until everything ticks like a clock. TQM and ISO 9000 adherents can't get enough of this stuff.
"The CMM was designed to help guide improvements in software organizations that continually miss schedules, overrun budgets and deliver defective software. Finally, it was designed to help software organizations consistently achieve quantitative business targets for quality, development time, cost and so on..."1
"What we have here is continuous demands for improvements in productivity. One of, and maybe the best way that we know as an organization of how to drive improvement in the software development process is to really deploy an SEI defined process. I don't know if it's the best, but its the best way that I have found to do it ... (data indicating that this is the right way) is pretty meager and I wouldn't call it compelling..."2
In the other camp are the people who build systems for a living, and have accepted that developing systems is a unpredictable, chaotic business. This camp believes that software development is an empirical process that requires significant thought and cannot be canned,
"...you can't follow a recipe without having to think. Some methods claim to fully automate the process, to tell you every step to follow so that software design is painless and faultless. They are wrong..."3
"Our attempts to use the Waterfall Methodology, an approach which assumes a fully constrained development environment, has been a fools errand. We have been embarked on a mission to do the mathematically impossible! It is not surprising that a third of development projects are canceled before completion and that those which are completed are, on the average, 189% over budget."4
"I believe that creating software is very complex; we cannot simplify software development by imposing simple models on it."5
The empirical camp believes that Texas Instruments is doing the right thing by trying to improve the process; however, they believe that TI is trying to improve the wrong process.
The debate between these two camps has heated up during the last year, with major bombardments in the press and on the Internet.
The debate is symptomatic of the ailments occurring when development teams try to build client/server and, particularly, object-oriented systems. The empirical camp is attempting to ensure that object-oriented technology doesn't lose its potential productivity by being encapsulated in the SEI defined approach, and inheriting the failed legacy of CASE.
Every debate needs to refer to first principles. Without this, what fun is there? In the case of the people that claim systems development is unpredictable and chaotic, the first principles from process engineering have been used.6 The question, Defined or Empirical, was put to process automation scientists at DuPont's Advanced Research Facility. They inspected some of the leading systems development processes and concluded that it is no surprise that most development projects don't reach their planned conclusion. Systems development is an empirical, unpredictable process.
Applying concepts from industrial process control to the field of systems development, methods can be categorized as either "defined" (theoretical) or "empirical" (black box). Correctly categorizing a process is critical to its successful implementation.
Models of defined processes are derived from first principles, using material and energy balances and fundamental laws to determine the process model. For a systems development process to be categorized as defined, it would have to conform to this definition.
Models of empirical processes are derived categorizing observed inputs and outputs, and defining controls that cause them to occur within prescribed bounds. Empirical process modeling involves constructing a process model strictly from experimentally obtained input/output data, with no recourse to any laws concerning the fundamental nature and properties of the system. No a priori knowledge about the process is necessary (although it can be helpful); a process is treated like a black box.
Upon inspecting the systems development process, it became obvious that :
The scientists were particularly amazed at our industry's failure to use effective controls on our empirical process. Instead, we have followed pert charts believing that a system would result. For empirical processes, they said, use enough controls to know what's going on; then you can react to unpredictable results at any stage of the process. Know the risks and actively control them.
Many methodologists agree with this assertion; "...you can't expect a method to tell you everything to do. Writing software is a creative process, like painting or writing or architecture... ... (a method) supplies a framework that tells how to go about it and identifies the places where creativity is needed. But you still have to supply the creativity...."7
This discussion became quite heated at the recent International Conference on Software Engineering in Seattle. Michael Cusumano8 presented findings regarding the Microsoft development process, which is anything but defined, and certainly doesn't qualify for anything good in the SEI CMM scheme of things. Grady Booch, talking for the SEI CMM side of the debate, weighted in with, "If I had a few billion dollars of cash in the bank, improving my software development process would be far down my list of things to do ... most software-intensive organizations eventually live or die based upon their ability to follow through with a sustainable and mature software development process."9
Appropriate responses include :
To get out the message of how they successfully build systems, a number of ISV's are making their empirical development processes and control tools available on the market to other development organizations (including IS shops). VMARK and Borland are particularly aggressive - they want to ensure that their OO tools generate the productivity benefits they should. Both are working with Advanced Development Methods on a SCRUM empirical development process ... an iterative, incremental approach that expects and controls the unexpected. Risk, Issue, Backlog, Problem and Change control are an inherent part of SCRUM.
An overview graphic of the SCRUM development process looks like:
Other forms of empirical, chaos-tolerant development processes are used. Quality control and testing are built into the entire development process by Software Testing Laboratories, whose customers include the largest ISV's, including Microsoft. A fractal approach to empirical development is described very well by Electronic Data Systems Research10. Phrases like "good enough software11" and "the best possible software" help describe these empirical development processes.
The applicability to IS shops of the empirical processes, with controls, used by leading ISV's was studied by a leading technology research firm, Aberdeen Group. In an Aberdeen Group Product Viewpoint12, the following question was asked :
"Enterprises depend upon their custom applications to provide core competencies in critical areas such as improving customer support and creating more efficient business processes. Likewise, ISVs rely upon their packaged applications to create value for their customers and therefore revenues and profits -- their very existence is dependent upon the success of their development efforts. Yet the acceptable, traditional way to implement mission critical business applications is very different from the way ISVs bring product to market. Why? "
Aberdeen determined that :
"New database, hardware, application, and networking technologies are not proprietary -- your competitors can use them as effectively as you. ISVs understand this, and bet on high-speed, quick-U-turn development processes to succeed in cutthroat markets. IS organizations can take the same tack: learn from ISVs and emphasize speed-to-deliver and flexibility rather than costs and backward compatibility. Methods like ADM's SCRUM give IS a way to drive ISV best practices deep into the development organization.
At the same time, IS should not have to be on the bleeding edge of yet another new software development methodology. Enterprise IS is not being a pioneer by following the SCRUM methodology. 3/4 of the ISVs are already using it. ADM is simply making it available in the form of the Product Management Facility.
Most importantly, IS should consider where this is leading in the long term. The new mission-critical applications can give the enterprise a competitive advantage -- but can the IS organization keep delivering after the first home run? ISVs that stay around for more than 10 years don't rest on their laurels: they use rapid development and flexible processes to gain more and more competitive advantage. Methodologies like SCRUM aren't just a passing fad or one-shot fix. If IS wants to succeed in the long term like the best ISVs, it should take a good hard look at ideas like SCRUM."