Our approach to the process used to develop technical specification in a complex regulatory or standards base domain includes appropriate, timely and effective inspect and adapt cycles. Software development professionals who believe in the principles of Scrum, Agile and XP often point to issues relating to up-front design by discounting the ability of the design team to create suitable design, architecture and specification at a point in the project where the team knows the least about the project. They also often point out that the creation of a comprehensive technical specification is inherently wasteful and actually delays the real start of the project, thereby delaying the delivery of business value. While there is significant validity to these statements, we believe that a pure story driven approach is also flawed. The originators of eXtreme Programming often are accused of relying totally on the TDD process to evolve architecture and design. While many XP teams used this as an excuse to do not up-front planning the true originators did have a basic model that was developed early in the development phase. However, all too often this model was not updated and became outdated and useless.
Identification of Technical Risks
They key in an effective Agile approach to complex project design and architecture is to provide and utilize inspect and adapt cycles for the design to grow as the development progresses. Some initial analysis and design work is required to prime the process. In most instances this process starts with the analysis of potential technical risks in the project. This can include items like new or updated languages, new or updated platforms, new integration points or methods and standards or regulatory compliance requirements. This initial phase may include some investigation of the new or unique areas in the form of samples, spikes and prototypes. However, reliance on these methods is also rarely sufficient.
Initial Architectural Design
The next step is to define an initial high-level architecture. Traditional approaches to architectural design are often focused on classification and organization of system components. This can be done relatively well where the problem lends itself to a layered design where the logical layers coincide with the concepts in the problem domain. Other approaches include the “nouns and verbs” method or tall, inheritance driven, Object Oriented designs. There are several other similar approaches that result in designs that are difficult to understand and maintain, use multiple inheritance, are too tall from an OO perspective or only seek to make functionality easy to find in large codebase.
We believe that architecture should be more about identifying the major areas of concern or concepts in a system rather than creating an organizational scheme. The goal should be to separate these concepts so that as the design evolves or new requirements emerge, they can be accommodated by adding to the solution rather than changing previously developed components.
Once these concepts are identified and briefly described, the relationships between them will begin to be discernable. Many of these relationships will be recognizable as examples of or similar to software design patterns described by the leading experts in the discovery, description and use of software design patterns.
Care must be taken to not add too much detail at this point as the team has only begun to identify the problem domain and it is very easy to define a system where tall class hierarchies are used for specialization of behavior.
The appropriate use of the abstractions identified in the discovery of the concepts can lead to a more maintainable, flatter class hierarchy that is easy to extend while preserving understandability and maintainability.
Identification of Appropriate Layers and Layer Interfaces
Once this initial design is defined it is often appropriate to identify the logical and/or physical layers or workflow steps in the solution. This often results in the grouping of concepts into these conceptual layers. Caution must be exercised to leave the design open to extension through the appropriate use of abstract classes and Interfaces. Here again software design patterns will be identified. Particular attention should be given to the programming interfaces between the concepts and layers that have been identified. These interfaces should be simple yet sufficient.
As these concepts and layers are identified, it is very beneficial to be aware of the concept of dependency inversion. Simply stated, the service class should be dependent on the needs of the client class for the interface of the service class.
Prioritization of Target Scenarios to Mitigate Risks
At this point the analysis can turn to the target scenarios that will help mitigate the risks identified. These scenarios would be packaged into an early release of minimal functionality intended primarily to provide a vehicle for mitigating the top risk issues.
Development of Initial Product Roadmap and Release Plan
Then the initial Product Roadmap and Release Plan can be started. This analysis only needs to be at a high level and should only commit to the customers and/or stakeholders the features that are truly must-have for a minimum release. The goal should be to release the smallest set of features possible that deliver any business value. We recognize in a standards or regulated environment, this may be more than we would normally recommend, but the traditional approach of trying to get absolutely everything in a release in still inappropriate.
The Delivery of an Increasingly Capable Product
In most cases, the calendar time to reach this point should be a few weeks and the cost should be significantly less than a traditional full analysis and specification project. Yet the risk will be significantly reduced. This Product Roadmap and Release Plan should identify a minimum scope for initial commercial or internal viability. This set can be used to set the initial budget for the project, which should be significantly less than the budget for release of a traditional project. In fact, in many cases a large proportion of the anticipated full functionality can be delivered for less than half the traditional initial budget.
The process of identifying the next set of minimum, focused, value centric features continues at the fastest pace possible, consistent with the ability of the target market to accept and deploy low risk, incremental deliveries.
More Business Value, Less Business Risk
Scrum and Agile are about the mitigation of risk in software product development. It has been proven many times that the traditional approach of trying to specify and estimate the project in project approval phase leads to poor results, especially in domains where technology and markets are changing rapidly. The approach we use will enable an enterprise to deliver significantly more business value faster with much less waste and risk compared to traditional methods. This is not a simple transformation for the existing enterprise and almost always requires outside help to be successful. However the market advantage gain can be enormous.
