Where's the value estimate?
Although the idea is simple and compelling, I've never seen it being used in practice. Or to be more precise, I've only seen half of it being used. In other words, I've been in projects where the developers or other people try to estimate the time (cost) it takes to build something. However, I have never been in a project where the value estimates have been made for the backlog items with same accuracy. So, why to bother estimating the cost, if we don't estimate the value?
Probably the reason why the value is rarely estimated, is simply because it is more difficult. It's often quite easy to say that one feature is more important than some other feature and provide convincing arguments. However, saying how much more important in terms of money is much more difficult. I guess this is because features usually are a part of something bigger and it can be really hard to track their effect on revenues even afterwards.
Simplified ROI
So how can we make proper prioritisation if we don't know the exact value that features bring? One solution for this is to simplify the ROI formula. Please consider what happens if the cost is 1, i.e. if all backlog items are approximately equally sized. The formula becomes ROI = (value - 1) / 1, which is basically same as ROI = value. This means that we don't have to know the exact numbers anymore. It's perfectly fine to say that this feature is more important than some other and make the prioritisation decision only based on that.
Obviously this works only if the development team knows how to split the tasks into small enough pieces. If your team has tasks that take 1 day and tasks that take 1 month, you cannot use the simplified formula. But if all of your tasks basically take 1 day, you can just focus on the value the tasks produce.
You may say that the formula cannot be simplified like that if tasks sometimes take half a day and sometimes 1.5 days. The latter is three times compared to the former, right? But I think that in our highly variable software development context it's ok to estimate those tasks as an average of 1 day. Although we might have an idea that something takes half a day instead of one, the experience has shown at least for me that those estimates aren't very reliable. So from the estimating point of view, using the average of one day is just fine.
PS. You may want to read my comment from the comments section. It hopefully clarifies this post somewhat.
Hi Henri,
ReplyDeleteNice post. I like the simplified ROI.
We definitely see businesses who are very unwilling to put any number on value, while still expecting a high degree of accuracy in estimates, and even using them as a beat-up tool later on...
I would prefer 'Value' to 'ROI', as ROI is not the only useful measure here - for example Cost Of Delay is often more useful as an input to prioritisation.
With one client, we do t-shirt sizing of both Value and Cost. We do this in a very lightweight way, with an assumption that we will mostly be wrong, but that it will still help our decision making.
Then if Value is 'L' and Cost is 'S', it's a no-brainer, and the work gets prioritised according to value without further estimation.
If Value is 'S' and Cost is 'L', then it's not going to happen. If there is still a need for the value, we may use tools like impact mapping to identify an alternative route to achieve it.
If it's tighter than this, then we may need to do further work to refine both the Value and Cost estimates, to determine whether there is a viable approach.
Pete.
Henri, if the backlog contains equal sized items, the cost of implementing thme is only "1" if that unit is the actual cost of implementing each of them. The cost is the sum of all the backlog items x cost. If there are 30 items, with cost of 1, then the cost is 30 units. You didn't say what the units are.
ReplyDeleteAnd the formula you've used in not the ROI formula. ROI = (Value-Cost)/Cost, If "all" the back log cost equal 1, what are the units of measure for the Value.
If you esgtimate all the tasks and take their "average" you'll be ignoring the variance in those samples, which means you're ignore the variance in the ROI.
This is simple bad math and worse management.
I'd be interested in where a non-trivial - actual SW project has "same sized" backlog costs.
Glen, thanks for pointing out those issues. After I finished writing, I felt like I couldn’t be clear enough with my thoughts. So your comments hopefully help me to clarify my ideas.
DeleteFirst, the unit is some amount of money. For example if the average time for one task is one day, then the unit is the cost of one day. Then obviously also the value unit is money. But if we can simplify the formula, the absolute amounts for value do not matter. It’s just enough to have an idea which tasks bring more value than the others.
If the backlog contains 30 items, I would only consider max ten of them at once. If some task has initial priority number 25, we shouldn’t start doing it, no matter how small it is. Also we shouldn’t spend too much time on tasks that won’t be done in the near future because our preferences are very likely to change before we get there. Those ten items I would split. You are right that they rarely are of same size but we can split them so that the new items will be closely enough same size.
The splitting does not obviously remove the variance but I claim that it can remove the variance so that it’s not relevant anymore. I mean that surely we can spend time on estimating those pretty small items but due to the variance, I’m not sure if it’s worth it. In other words, when the tasks are on average small enough and the variance is not too big, we can say that all the tasks have size of ~1. Spending time on detailed estimating so that we would get the perfect backlog order just isn’t worth it.
I would like to emphasise that I’m not talking about project or program level ROI here. I’m talking about ROI in team level and how useful that tool is in practice when backlogs are prioritised. The idea makes sense but if it is really hard to implement in practice, then there’s something wrong with it and we should try to find better ways for backlog management.
Thanks for the clarity. I assumed it was the project or even the business ROI rolled up from the investment in the labor for the project.
ReplyDeleteNow when you said it, I realised that it probably wasn't very obvious from the text. Thanks for your comments again, Glen.
Delete