James Shore wrote an interesting blog article called "Voluntary Technical Debt". He and Dave Woldrich are developing a commercial service called cardmeeting.com to support distributed agile teams. Shore describes how he and Dave cut corners with their initial implementation, intended to be a "spike", because of time pressure to demonstrate the software at the Agile 2006 conference. In other words, they intentionally created technical debt to meet a marketing deadline. The team is now taking the time to repay that technical debt — with interest. Shore reports that the debt has caused delays in developing new features but he believes he made the right decision.
I’m not questioning whether this was the right decision or not. However, this is a classic development pattern for non-agile teams. The managers (who are the same as the development team in Shore’s case) define a marketing event as a development deadline. The developers decide to build a prototype or spike solution to minimize technical risk when developing the production code. The managers see the prototype and declare it "good enough" to serve as the foundation of the production code. At that point, the developers are pressured into creating relatively low quality code to meet the deadline. There are a few differences between the cardmeeting.com scenario and the more common scenario. First, the cardmeeting.com team apparently doesn’t have another impending marketing event so they can invest the time to pay back the technical debt. Secondly. they are a very small team with a management team (themselves) who understand the importance of paying back the debt. Most teams are not that fortunate.
I think Shore’s article describes an excellent example of what happens when agile idealism meets the everyday realities of software development. I greatly appreciate that he has written about that experience rather than quietly covering it up. What are some conclusions we might draw from that experience report?
- Agile practices slow down development. At least, it appears that the cardmeeting.com team believed this was true in the short term. I believe agility is not primarily about speed, but about the ability to effectively respond to change and uncertainty. A near term reduction in development can be a good investment for increasing long term project productivity. However, if there are near term deadlines, or a long series of near term deadlines, the forces against using agile practices can be difficult to resist.
- Paying back technical debt requires "interest" payments.. This has also been my experience. Shore isn’t very specific about what type of interest payment is required for the debt. What I’ve seen is that it’s more difficult to write tests after the production code has been written. With a small team, paying off the technical debt means that few or no new features are developed for some period of time. This can hurt the forward momentum, motivation, and focus of the developers. There’s also marketing interest to be paid. Potential customers tire of waiting for new features that important to them and lose interest or go to a competitor. It can be difficult to win back customers once they have formed a negative opinion.
- When Agility and Reality clash, bet on Reality to prevail. Yes, this will probably be a controversial statement to the extent that "Reality" is at least partially subjective. However, the reality in this situation is that a team consisting of experienced agile developers (including a coach and award winning agile thought leader) made the same non-agile choices as many teams make in similar circumstances. And they believe those choices were correct (and very well may have been). It seems to me that people who sell agile approaches often believe the reason that teams reject their suggestions is because of ignorance, stubborness, fear or something similar. Often these teams reject agile practices (at least, in the short term) for the same reason as the cardmeeting.com team. They did it intentionally and believe it was the right decision given their specific context.
- Consider timing when introducing agile practices. The cardmeeting.com team is now paying back their technical debt. They don’t have another near term deadline so this is a good time to do it. Unfortunately, for some teams, there appears to never be a right time and the debt continues to increase.
I recommend reading Shore’s original article. I’m also interested in hearing about other people views of what happens when there is apparent clash between Agility and Reality. I should also be clear that this contrast between agility and reality is not a criticism of agility, but about the more extreme forms of agile idealism. I’ve used similar comparison between reality and the prevailing development strategies on several teams to convince the management to adopt more agile behaviors rather than continuing with their current development idealism (waterfall/phasist, management by wishful thinking, or whatever).