Constructing Agile Processes

A Software Development Process usually cannot be successfully applied out of the box. Practitioners know that methods have to be adapted to company culture and project constraints. However, many current Agile Processes lack an explicit description of the constraints, alternatives, and side effects of such changes.

In this article from 2002 Hasko Heinecke, Christian Noack and Daniel Schweizer describes how a pattern language with clearly described forces and trade-offs can be derived from existing development processes, and how the result can be used for a Bottom-Up approach to process adaptation.

Constructing Agile Processes

In the past couple of years, Extreme Programming (XP) and other Agile Software Processes have started to gain considerable impact on software development in enterprise IT departments. Nevertheless, still many project managers are reluctant to switch to these new methodologies. They feel that some of the practices and techniques employed by Agile Processes are too alien to their companies’ cultures. This is the reason why in many projects processes are not applied “by the book” but rather are adapted to the specific needs and standards of the environment.

However, existing Agile Processes usually do not specify explicitly how to be adapted. Aside from that, it is not easy to decide which of the growing multitude of Agile Processes to deploy in the first place. It requires much experience and a thorough understanding to select and tailor a new process to meet the specific requirements and constraints of a given software project.

The problems that need to be addressed by software processes are complex and cannot be solved without taking into account the concrete environment in which they are implemented. Pattern languages are an instrument to describe possible solutions to such complex problems in a structured way.

[A] pattern is a three-part rule, which expresses a relation between a certain context, a problem, and a solution.” [Ale79, p. 247]

Pattern languages require the relationships between the patterns and the environment to be made explicit. Software process descriptions would benefit from this because the constraints for adapting them would be obvious.

Therefore, we believe it would be worthwhile to rewrite software processes as pattern languages. Moreover, if those could be integrated into a single comprehensive pattern language, we would have an alternative to adapting processes: Software processes could be constructed, using the concrete project-specific constraints as input.

Deriving the actual pattern languages for each of the Agile Software Processes is beyond the scope of this paper. Instead, we give some examples of patterns derived from different processes and show how they can be integrated, to become part of a common pattern language. Furthermore, we outline how such a common pattern language can be used to construct a custom process in a Bottom Up fashion.

Deriving the patters

The foundation for the construction of custom software processes is a comprehensive pattern language. Its building blocks are patterns. Deriving them from existing processes suggests itself. An important criterion for the quality of a pattern is whether it has been applied suc cessfully more than once. Therefore, we chose examples from software processes that have been proven in industry use.

The actual procedure of deriving patterns from a software process is interactive rather than formal. E.g. Christopher Alexander describes how patterns are conceived in an informal conversation (see [Ale79], p. 270ff). To the best of our knowledge, there does not even exist a formal procedure. We believe, professional judgement and discussion among peer is the only feasible foundation for such work. However, the focus of this paper is on the integration of patterns from different processes, and on constructing processes bottom-up. It is out of the scope of this paper to describe in detail how we arrived at the example patterns we use. This is not a trivial task, though, and will be a subject for other papers.

Software process literature is usually not already in pattern format. Where we find pattern-like structures, such as the core practices in Extreme Programming, they do not explicitly mention constraints and relationships to other elements. Other patterns can be found which are not primary elements of the process in question. E.g., eXtreme Programming requires the project team to work in a common office. This in itself is not one of the core practices, although it is understood by the XP community to be an important property of XP projects. In fact, a lot of discussions in newsgroups and on conferences concentrate on this topic: Is a common office necessary to do XP properly, and what if a common office is not feasible? Consequently, special care must be taken when deriving a pattern language from a process description to gather all the implicit patterns.When deriving a pattern language from a process, it is necessary to define a pattern structure first. In the following sections we describe and justify a possible structure that we found useful. Here is an example pattern we derived from XP:

The example displays the general pattern structure that we use for collecting process patterns. The minimal set of features of a pattern are the name, the problem addressed, the constraints, that define the context in which it is applicable, and the solution to the problem. (See e.g. [Ale79], [G+95]) As can be seen in the example, our pattern structure adds some elements.

Different processes can use the same name for practices that are similar but still not identical. Therefore, when we derive a pattern we add the source process name to each pattern description.

We found that most patterns that can be derived from existing software processes actually address more than one problem at once. (Conversely, every problem is usually addressed by more than one pattern.) Therefore, the problem section of our pattern structure is a list of problems rather than a single problem.

The list of constraints is actually broken into two sections. The first is named “constraints” and lists the general preconditions and restrictions for the application of the pattern. The second is called “implied patterns”. If one pattern requires another one to work, we call this an implied pattern. It is a real constraint, but of a special nature. We discuss the consequences of implied patterns
in the section Constructing Processes below.

The solution part of the pattern often contains just a reference to the original process description. Only if the pattern is not obvious or not sufficiently described there, we add some explanation of the solution itself.

This pattern structure we found applicable to many different software processes. Below is an example from the Capability Maturity Model (CMM), a process model believed to be non-agile by many.

Using this structure, each software process can be formulated as a pattern language. The individual languages (that describe one process each) will overlap, but they are still separate. In the following section, we will describe how they can be integrated into one common pattern language.

Integrating the pattern languages

Each eligible software process will be refined into a separate pattern language. This in itself provides already valuable information about the patterns and their relationships therein. However, our goal is to provide a solid foundation for constructing agile processes. So the multitude of pattern language must be integrated into synthesis, a single, comprehensive pattern language.

One important source of differences between processes is the vocabulary. In order to unify them, it is therefore necessary to unify the vocabulary first. Since all software processes address a similar set of problems — after all it is always some kind of program that has to be developed and delivered — we suggest the collection of all problem descriptions from the pattern language in a catalogue.

This catalogue will be the starting point for process construction, as we will show in the next section, so particular care must be taken to make it complete and consistent. By complete we mean that all the problems addressed be a process must be included in the catalogue. By consistent we mean that problems should rather not overlap. Often, we find problems that are similar but not identical. It is then necessary to isolate the similarity into a new problem description to separate it from the two original
problems.

Sometimes one of the original problem descriptions is then completely encompassed by the new one, while the other one is a superset. But equally often, we end up with three problems where before there were only two. If that happens, we have to go through all affected process descriptions and review them with respect to the new problem. Quite often, we find that patterns will have to be restructured as a result, and new patterns are found as a result. This is an iterative procedure that may have to be repeated for each of the problems of the respective pattern languages. It is tedious work, but it results in a much better (and much better documented!) understanding of the respective software processes.

When a complete and consistent problem catalogue has been built, the next step is to repeat the process of integration and unification for the pattern constraints and the patterns itself. This, however, is easier because there will be much less overlap. Certainly, processes use different terminology in these aspects as well. But since a lot of the constraints were formulated during pattern derivation rather then taken from the process descriptions, they are generally much more organized already.

Finally, we arrive at an integrated pattern language with the following characteristics: The problem descriptions are clearly formulated and separate. The constraints are unified and implied patterns are clearly marked. The patterns themselves define a common vocabulary. We further build three indices: 1) A list of the pattern names to be able to find the patterns in the catalogue; 2) a list of the pattern sources, to be able to find all patterns from one process; 3) a list of the addressed problems. The last index is the most important one for constructing processes but also the most difficult one to obtain. To create this index a list of key words must be build up, and all patterns must be harmonized, so that equal problems have equal problem entries in a pattern. To be able to construct a process from the catalogue we suggest to generate a dependency graph containing all filed patterns and showing the implied patterns.

Contructing the process

Within most of the projects, processes are not applied out of the book. We conducted several interviews with project teams who claimed to use Extreme Programming. We found that most of them did employ certain techniques like Unit Testing, but left out others, like Customer on Site. This kind of adaptation was born out of necessity rather than ignorance: The circumstances simply did not allow to have a customer on site. Different projects also applied different changes to the process.

However, these adaptations are frequently done without control: First, the existing techniques within the process and beyond it are often not taken into concern, because they are not well understood. Second, the existing problems are not taken into account while adjusting a process to the specific concerns because there is no checklist or catalogue to compare against. Third, the dependencies between the chosen techniques are not clearly documented. This is where a pattern language helps. It can be used as a construction kit to build up a process.

To choose the building blocks of your process you first have to think about the expected problems you will have to handle during the whole process. Most of the problems are very similar for almost every project and are already known. You can take them from the catalogue of problems in the pattern language. By integrating more than one process into one pattern language, even rare problems or such that are overlooked easily are provided in the form of a checklist from which you can select the problem areas that apply to a given project.

With the resulting collection of problems you can start selecting the patterns from the catalogue and build up your process. This is a creative work. Which patterns are actually chosen depends highly on the preferences, the existing technical and social skills of the people and several other circumstances of your project. The pattern language helps not to forget the options you have, and it helps not to lose the focus on your problems. It also helps ensure the consistency of the resulting process, by clearly documenting relationships between patterns, namely those that are implied by others. When all problem areas a covered by patterns, the process is complete.

The final question that remains is, in what way are the resulting processes agile? While it is beyond the scope of this paper to justify a formal definition of “agileness”, we give one that covers what we find important in Agile Processes: 1) They stress importance of working software rather than that of process artefacts. 2) They consist of a minimum number of process elements. 3) They address
the problem of rapidly changing requirements.

So on the one hand it is certainly possible to create “Big M” methodologies with a pattern language as the outlined in this paper. On the other hand, it does give project managers a tool with which they can create processes that consist only of a minimum number of patterns. When these patterns address the rapid requirements change problem (from the problem catalogue), then the resulting
process should be reasonably agile.

References

• [AIS77] Ch. Alexander and S. Ishakawa and M. Silverstein: A Pattern Language, Ox- ford University Press, New York, 1977
• [Ale79] Ch. Alexander, The Timeless Way of Building, Oxford University Press, New York, 1979
• [Amb98a] S.W. Ambler: Process Patterns: Build- ing Large Scale Systems Using Object- Oriented Technology, SIGS Book, Cambridge University Press, New York, 1998
• [Amb98b] S.W. Ambler: More Process Patterns: Delivering Large Scale Systems Using Object-Oriented Technology, SIGS Book, Cambridge University Press, New York, 1998
• [Bec00] K. Beck, Extreme Programming Ex- plained: Embrace Change, 2000, Addi- son-Wesley
• [Cop95] J.O. Coplien: A Generative Develop- ment-Process Pattern Language, Pattern Languages of Program Design, 1995, Addison Wesley Longman, Inc., pp. 183-237
• [Cun96] W. Cunningham: EPSIODES: A Pat- tern Language of Competitive Devel- opment, Pattern Languages of Program Design 2, 1996, Addison-Wesley, pp. 371-388.
• [G+95] E. Gamma and R. Helm and R. Johnson and J. Vlissides: Design Patterns, 1995, Addison Wesley
• [P+93] M.C. Paulk et al.: Key Practices of the Capability Maturity Model, Version 1.1

---

• ← Previous Post
• Next Post →