Friday, February 01, 2013

Why I hate spikes

The recording of Dan North's recent talk on "Embracing Uncertainty" set me thinking again about what I like and dislike about agile methods such as scrum. As we end the 157th consecutive sprint of the Virgo project, I certainly appreciate the focus and sense of making progress that each sprint brings.

But what I really dislike are spikes.

The principle of a spike sounds fine: carve out some time to do some exploratory work. But I soon find myself too constrained by a spike. One definition of a spike is (emphasis added):

"a story that cannot be estimated until a development team runs a timeboxed investigation. The output of a spike story is an estimate for the original story."
There are some assumptions buried in that definition which I think show some of the problems:

  • implicitly, and significantly, a spike is something you do in a sprint
  • timeboxed - of course we can't commit an indefinite amount of time to a spike, but a timebox kind of implies that at the end, you'll come up with "the goods"
  • estimate - the "goods" are an estimate
  • original story - there's an assumption that the spike won't change your perception of what the story should be.

Here's where Dan's presentation helped me understand what was going on. Spikes are an attempt to fit some explorative work into an agile process which is designed for repeatedly churning out reliable software with minimal risk.

My natural way of doing exploratory work is rather different. First of all, I don't presume to know what it is I expect to find. I have some ideas to try out, some questions to answer, or maybe just something I need to learn about (and believe me, the longer I stay in this job, the more ignorant I realise I am).

So the actual "goods" are really answers, understanding, and more ideas. It might be possible to turn some of these into estimates for a story, but that's not really the goal.

What's wrong with trying to fit exploratory work into a sprint, you may ask? I think the psychology of reporting status in a stand-up meeting each day forces me to focus on things which will deliver immediate results. Sometimes the best way to explore something is to follow a number of lines of inquiry, get yourself thoroughly overwhelmed, take a break, and then (after several pennies have hopefully dropped), start to understand the underlying concepts.

Worse still, if I have at least one non-trivial task in the sprint besides a spike, it's really hard to get into the exploratory frame of mind. The only way I've found is to get the other stuff out of the way as quickly as possible and then try to switch modes.

So in the future, I propose to approach exploratory work differently. I'll treat it as something that's done outside a sprint. I'll still keep the idea of a timebox - it's no good going off indefinitely and losing contact with the rest of the team. But I definitely won't require that the output is an estimate of a story.

If this sounds too risky, then good. Innovation is supposed to be risky.


J. B. Rainsberger said...

I see a gaping logic error here. "Spikes are an attempt to fit some explorative work into an agile process..."

Yes, and no. Spikes are an attempt to explore a specific kind of exploratory work, namely the one you cite, and not an attempt to do *any kind* of exploratory work. In fact, *slack* is in part an attempt to make room for general exploratory work with, in part, the hope of innovation. This idea goes back to Connextra and "Gold Cards" over ten years ago. See slide 27 of

I see no inherent constraint or property of the two ideas that makes spikes and slack mutually competing on a project.

As you say, we have psychological barriers to putting slack in our project plans, and we might lie and call things "spikes" in order to justify the work, but that's a reason to dislike liars, or low-trust environments, but not spikes.

Glyn said...

I've never heard of Connextra, Gold Cards, or for that matter "slack" in this context, so thanks.

Aaron Digulla said...

The most important core value of agile is to put common sense over rules.

So, yes, agile comes with rules but the expectation is that you have a brain, know how to use it and apply the rules wisely. Rules only ever apply at most 90% of the time.

Or to put it another way: How can something be agile and 100% defined by rules at the same time?

So agile "rules" and "definitions" are just guidelines, things that worked well for other in the past. That doesn't mean they work well or at all for yourself. Which is why you have to pick those rules and definitions that work in your specific case. And sometimes, you have to sit back and think whether the rules you picked are still good enough to follow.

Glyn said...

That suits me. I guess we inherited an overly prescriptive interpretation of agile that we need to break out of without feeling guilty. ;-)


OSGi (130) Virgo (59) Eclipse (10) Equinox (9) dm Server (8) Felix (4) WebSphere (3) Aries (2) GlassFish (2) JBoss (1) Newton (1) WebLogic (1)