Friday, March 14, 2014

Need to estimate, part II

In the spirit of Need to estimate, the following discussion happened yesterday.

- I know that you don't like estimating but I'm afraid I have to ask you to do that. We need to know which of these (on the wall) will be done by the end of March.

- I don't have a problem for estimating if that is the valid way to solve the real need. So first I would like to understand what kind of decisions are made based on these estimates.

- Well, it's important to inform certain people if this (pointing to a task on the wall) won't be done because then they have to find another solution for the problem.

- Are you sure asking the estimates is the right question here? Isn't the right question, at least initially, whether we should build this feature at all? I think we talked earlier that it actually doesn't make sense for us to implement it since it kind of doesn't belong to our application?

- Yes, I guess you're right. And there will be so few users that it won't pay back if we build it. Manual work for these cases should be just fine, at least by now.

- Exactly.

- Ok, I will just tell them that this won't be done by the end of March and actually not at all in the foreseen feature. Thanks!

- Sure, no problem.

Thursday, March 6, 2014

Need to estimate

Before going any further, I would like to mention that Neil Killick has written an excellent post related to this same topic: People Need Estimates. You might want to read that post before or after my post, which tells one story where estimates were "needed".

We need estimates

Prioritization in our project was initially made by dividing the features into four categories: must, should, could, and won't have. Our team had basically finished all the must haves and since there was time left before the pre-defined launch date, the management wanted to prioritize the should haves.

The should have items had already been prioritized based on business value and the request for the team was very clear: the management needs estimates for all of the should have items so that they can make the final prioritization.

Or perhaps something else?

I already revealed above what the management actually needed. They needed to decide how the development team should spend the scarce time there is left before the launch. They didn't need estimates. Surely estimates is one solution for this problem, a very typical one actually.

So what's wrong with the estimates related solution? One thing is that if you don't estimate the value, estimating the cost is quite debatable. Another thing is that in this case the should have list was very long. If the management wanted the team to estimate the whole list, it would have meant that we spend several days on trying to understand what some of the items mean and how we should implement them. Or we could have just thrown the famous "educated guesses" without even knowing what we were supposed to build.

Input for management

I didn't like either of the approaches so we did something else. We had all the items on our whiteboard vertically placed in the initial priority order. We went through the list and placed each item horizontally in one of the three columns: simple, complicated or external dependencies, and very unclear or lot of uncertainty. This took about an hour of our time.

When we presented the revised list for the management, our message was the following:
  • The items in the simple column are such that you could move them up in the priorities if you like since they should be rather straightforward to implement.
  • The items in the complicated column contain more risk and you may want to drop their priority if you like.
  • If there are some really important items in the very unclear column, we should investigate them more so that we can understand them better. (But we shouldn't investigate all of them because that would really waste our valuable and scarce time.)
Besides that we told them that if they really need some numeric values, we can use our statistics about the development lead times. They weren't interested in the numbers, the table was just fine.

Decisions made

Probably the most interesting thing happened (or did not happen) after we had presented the table: our categories did not really have an effect on priorities. The items that were initially on the top had so clear business value so that they remained on the top although they were mainly complicated ones. The simple items did not get to the top even though there were - well, simple. And the management asked us to spend time on only one very unclear item.

This tells me the message that the estimates were initially asked basically just because that's how it has always been done. On the other hand, it tells me another message too. Our management (intuitively perhaps) understands that it shouldn't focus too much on the short-term development cost but rather on the long-term business value. This is something I really like.

Sunday, March 2, 2014

Do we need ROI in prioritisation?

Return on investment (ROI) is a tool used in accounting to measure investment profitability but also in software development to prioritise backlog items. The idea is simple. Let's define the value that a backlog item generates, estimate the investment cost, and then we can calculate its ROI: (value - cost) / cost. When we have these for all items in our backlog, we can sort them by ROI and build the items with highest ROI first.

Where's the value estimate?

Although the idea is simple and compelling, I've never seen it being used in practice. Or to be more precise, I've only seen half of it being used. In other words, I've been in projects where the developers or other people try to estimate the time (cost) it takes to build something. However, I have never been in a project where the value estimates have been made for the backlog items with same accuracy. So, why to bother estimating the cost, if we don't estimate the value?

Probably the reason why the value is rarely estimated, is simply because it is more difficult. It's often quite easy to say that one feature is more important than some other feature and provide convincing arguments. However, saying how much more important in terms of money is much more difficult. I guess this is because features usually are a part of something bigger and it can be really hard to track their effect on revenues even afterwards.

Simplified ROI

So how can we make proper prioritisation if we don't know the exact value that features bring? One solution for this is to simplify the ROI formula. Please consider what happens if the cost is 1, i.e. if all backlog items are approximately equally sized. The formula becomes ROI = (value - 1) / 1, which is basically same as ROI = value. This means that we don't have to know the exact numbers anymore. It's perfectly fine to say that this feature is more important than some other and make the prioritisation decision only based on that.

Obviously this works only if the development team knows how to split the tasks into small enough pieces. If your team has tasks that take 1 day and tasks that take 1 month, you cannot use the simplified formula. But if all of your tasks basically take 1 day, you can just focus on the value the tasks produce.

You may say that the formula cannot be simplified like that if tasks sometimes take half a day and sometimes 1.5 days. The latter is three times compared to the former, right? But I think that in our highly variable software development context it's ok to estimate those tasks as an average of 1 day. Although we might have an idea that something takes half a day instead of one, the experience has shown at least for me that those estimates aren't very reliable. So from the estimating point of view, using the average of one day is just fine.

PS. You may want to read my comment from the comments section. It hopefully clarifies this post somewhat.