Estimating like an Adult – What to Steal from Agile…

In Part 1, we talked about the primary problems with estimating software development accurately:

Your task will have hidden complexity you hadn’t considered, as a function of it being software

You will be given extra or unrelated work to do, and if you fail to track and communicate this, you will look lazy and incompetent

But we ended with the uplifting message, which is if you plan for the former (by multiplying out your estimates), and make sure you adequately track the latter, then your estimates – when taken in aggregate – will probably be ok. And by ok, I mean better than 90% of other developers.

But who wants to be a 10 percenter, when you can be a 1 percenter?

I also promised Agile was going to solve all your problems ever when it came to estimating. This may not have been 100% accurate. What we are going to do is look at a whole bunch of Good Ideas(TM) that the Agile folk stole, curated, and invented when it comes to estimation.

Humans are Weird About Time

It turns out that we humans get a little weird about time. However much you believe “six hours” is “just an estimate”, however much buy-in there is from the team about this, and however many times you’ve accepted that there’s always hidden complexity, if a six hour task takes eighteen hours, everyone gets a little squirrely.

Squirrely is bad, because squirrely is bad for morale, and squirrely is bad for Project Manager / Developer relations. Squirrely leads to developers trying to hide actual progress and injecting “slush fund” tickets in to the work list (Agile: Backlog) that they can use to burn time down against, and leads to them getting demotivated, which leads to them spending more time on Reddit, and it’s a vicious cycle.

The first rule of Agile Estimation Club is that we estimate in Story Points, not time, and the second rule of Agile Estimation Club is that no-one ever tries to work out the Story Point to hour conversion rate (a Story is a collection of tasks, or one big task).

Good Idea #1: Stop estimating in time, because everyone has deeply held beliefs about time.

Story Points

Story Points are a measurement of the complexity and tedium associated with a task. Complexity is important, because complexity hides other complexity, and tedium is important because no-one works effectively on boring tasks.

How do we measure Story Points? Simple. Story Points are measured in Story Points.

What this really means is we estimate tasks in relation to each other. Is this task about the same complexity and tediosity as that task? If so, you should give it the same number of Story Points.

It’s super-tempting when you get started to estimate things in ‘days’ or ‘hours’, and then just drop the units, and look, you have Story Points! Don’t do this: everyone will remember that 8 points is ‘really’ 8 days, and people will get squirrely. The whole point with Story Points is breaking the relationship – at an individual task level – between Story Points and chronological time.

Good Idea #2: Estimate the complexity and tedium of tasks in relation to each other, rather than in relation to time.

A Couple of Refinements

Many teams estimate using numbers from the Fibonacci Sequence. That is, you’re only allowed to estimate your tickets using one of:

1, 2, 3, 5, 8, 13, 21, 34, 55

Although, This Is Agile, and so you’ll get cretinous Agile Consultants telling you how you only estimate using the Fibonacci Sequence, and then hand out cards with ‘1, 3, 8, 20, 40, 100’ on them, because they’re nice round numbers *twitch* – actually this doesn’t matter, as we’ll see, but still…

The idea here is that is a task’s complexity-tedium index grows, you have to decrease the accuracy – that is: the more complex a task is, the less chance you have of accurately estimating it, and you should account for that. When torn between two numbers, go for the larger – you’ll usually be right.

Good Idea #3: The more complex and tedious a task, the less accurate your estimates will be.

Velocity

If there was no relationship at all between Story Points and time, then there wouldn’t be much point in estimating.

We talked a lot about why we estimate previously, and estimates in complexity and tedium are of no interest to the business as a whole. The business needs Gantt charts, it needs deadlines, and it needs the occasional project to go massively over-budget and over-time so that one-day Presidential hopefuls can swoop in and fix them.

Velocity describes the number of Story Points a team can deliver in an iteration, which is a fixed period of time (three weeks, for example). In mature teams, Velocity should be fairly stable, and should see gradual increases as a team removes impediments.

When you’ve estimated every task or Story in your task list (ie: you’ve thought about and estimated everything you need to do), and once you know your Velocity, then you can tell the business how long something is going to take. You know you get 100 points done every 3 weeks, there are 1,200 points, and it’s maths that even an Agile Consultant can do… What’s more, for the first time, the estimates are likely to be somewhat accurate.

Good Idea #4: Your estimates relate to time only in aggregate, and only based on previous experience.

Mixed Ability Teams

Development teams are rarely homogenous. Some developers are worth three other developers, and some are regularly commiting work that needs to be unpicked and undone by other developers. Some developers have an OCD-level attention to detail, and some a stalkerish devotion to Twitter.

This has potential to screw time-based estimation systems. Remember: people get squirrely when something quantified in chronological units doesn’t match up to elapsed time. If a Senior Dev estimated a ticket as three hours, and a Junior is about to start their tenth hour on it, that’s going to be demoralizing. And if a Senior Dev is about to complete their third ten hour task in under an hour, they may get to thinking it’s time to see what’s new from Horse Ebooks.

When you estimate Stories in terms of each other, this largely goes away – people stop estimating Stories based on how long they think it will take them to do, and instead base them on how relatively difficult they are. This is a fundamental win, which I’m struggling to adequately do justice to here.

Good Idea #5: Estimating stories in Story Points and using Velocity for estimtes accounts for mixed-ability teams. This is huge.

Planning Poker

This brings us neatly to our final point, which is to do with the actual estimation process itself. I have the attention of a gnat when I’m not talking, which makes meetings at which I can’t be talking most of the time a nightmare for me (and ones when I can a nightmare for other people). Estimating meetings fit this category, as estimating is meant to be collaborative.

Everyone in the team should get their say when it comes to estimating, especially as they may end up doing the Story or task at hand. What usually happens, though, is that one person dominates, lays down their opinion, and everyone else plays Angry Birds on their phone. This is probably a bad thing.

The process of Planning Poker is: everyone has a deck of cards with the Fibonacci sequence (or some pretty, incrementing numbers labelled as the Fibonacci sequence *twitch*) on them. Each Story is discussed, and then everyone lays a card, face down, on the table with their estimate. Once everyone has selected a card, everyone turns them over at once.

You ask the person with the highest card why they think it’s such a complex task, and the person with the lowest card why they think it’s relatively simple. Anyone else can air their views too. Rinse and repeat until a consensus is reached.

This makes it very difficult to play Angry Birds.

If the team has been discussing what actual work a Story entails, and you haven’t been paying attention, you’ll struggle to put a sensible estimation down, and then you’ll be asked for why you chose such an estimate. If you guessed, rather than estimated, you may struggle with this, and your Project Manager will start to notice. With a persuasive (or sadistic) enough Project Manager, this leads to everyone being engaged.

Good Idea #6: Build shared ownership and engagement in the estimation process by estimating independently, then discussing collaboratively.

Summary

We’ve covered six good ideas from the Agile Estimation Process:

Stop estimating in time, because everyone has deeply held beliefs about time

Estimate the complexity and tedium of tasks in relation to each other, rather than in relation to time

The more complex and tedious a task, the less accurate your estimates will be

Your estimates relate to time only in aggregate, and only based on previous experience

Estimating stories in Story Points and using Velocity for estimates accounts for mixed-ability teams. This is huge

Build shared ownership and engagement in the estimation process by estimating independently, then discussing collaboratively

We haven’t covered all the intricacies of how to get started though. I suggest you start by Googling a list of Agile-related buzzwords, and seeing where that gets you, if you’re interested in a step-by-step guide.

Good luck, commander.