It often starts as a seemingly plain training request. Having decided to go the agile route, a client would like Cutter to train a certain number of employees in one agile method or another. We collect data on the demographics of the target population: architects, UI designers, product managers, project managers, developers, testers, and so on. We then move on to discuss the way these folks are geographically dispersed and what the team structure for the launched agile teams will be. Once these parameters have been nailed down, it largely becomes a matter of figuring out the logistics for training and coaching. A fairly straightforward process for rolling out the agile process, one might say.

The catch, time and time again, is that the problem to be solved had been defined from the outset in too narrow a manner. Naturally enough, a client wrestling with software challenges tends to gravitate toward improving the software method(s) in use. It is a small step from here to the conclusion that effective training and coaching are the only services that really need to be provided by the consultant(s).

The typical outcome of such a narrowly defined engagement is improved tactical agility. The teams become quite good at responding to frequent change needs. Stories get added, modified, deleted, or reprioritized fairly effectively and (many times) efficiently. The improvement can be assessed, and very possibly quantified, in quite a few ways. Yet, it is quite difficult, if not impossible, to point out the actual business benefits. The improvement in the process is clear enough to anyone who cares to examine it, but the end-to-end functioning of the software still leaves much to be desired. The “so what?!” question is typically raised under such circumstance: could it be the case that the resources put into the agile initiative could be better invested elsewhere?

My view on the subject — before, during, and after the engagement — is that an improved software process is a necessary but not sufficient condition for attaining the desired business results. Obviously, elaborate business plans are next to impossible to implement if the software process crumbles underneath them. However important tactical agility through a robust agile process is, it cannot on its own address strategic adaptability nor supply-demand alignment issues. To succeed as a business from a software perspective, you need all three: agility, adaptability, and alignment.

Lack of strategic adaptability is the end result of the accumulation of all “sins” that had been committed “against” the code prior to starting a new page with agile. The investment you put into improving the software process will probably pay off nicely in terms of quality, productivity, and time-to-market at some point in the future. Unless the level of your technical debt on the existing code is really low, however, your overall software system — old in combination with the new — will be slow to adapt. In addition to investing in agile, you will need to invest in pushing down the level of technical debt that had accrued over the years. You might choose to think of the investment in agile as primarily forward-looking. In contrast, cleaning up the technical debt mess might (erroneously) seem like backward-looking. However, both need to take place. As a matter of fact, doing the two together is quite synergistic, as technical debt is an excellent metric for driving agile software development.

If you step back to examine how the technical debt accumulated in the first place, you are very likely to find out that the vicious cycle depicted in Figure 1 has been taking place for years on end. It is not lack of expertise or the difficulty of development across the pond that got the software in trouble (though these might be contributing factors of secondary importance). Rather, it is the inevitable system dynamics that prevail when supply (i.e., team capacity) is misaligned with demand (i.e., requirements).



Figure 1 — The vicious cycle of technical debt.

Aligning supply with demand is in many ways the acid test for the long-term success of your agile initiative. Our knowledge of the agile process has increased dramatically over the past 10 years. Likewise, progress in technical debt measurement, reduction, and prevention has been quite impressive during the past couple of years. We know enough about agile methods and technical debt techniques to improve the state of affairs in all but the most desperate situations. As Figure 1 illustrates, we also possess a fairly good understanding of how misalignment could lead to the accumulation of technical debt of biblical proportions. What are often lacking are the discipline, will, and determination to force alignment.

My recommendation on supply-demand alignment in the sense defined in this article is to use the data structure of the enterprise depicted in Figure 2 as the tool for forcing alignment at multiple levels of abstraction. The principle is simple: the grand total of development and technical debt reduction work (probably expressed as dollars) at the epic level can’t exceed the amount allocated at the respective theme level. Likewise, the grand total of development and technical debt reduction at the feature level can’t exceed the amount allocated at the respective epic level. Ditto for the story level: the grand total of development and technical debt reduction at the story level can’t exceed the amount allocated at the respective feature level.

Figure 2 — Data structure of the enterprise.

You can opt to traverse the data structure of the enterprise in a top-down manner or in a bottom-up fashion. Whatever direction you choose, you must maintain the integrity of the calculations (i.e., the grand total in one level can’t exceed the respective allocation in the level above). Maintaining this integrity is quite possibly the number-one responsibility of the stakeholders you pick to govern the software process in your company.

— Israel Gat, Director, Agile Product & Project Management Practice