Need to estimate

Before going any further, I would like to mention that Neil Killick has written an excellent post related to this same topic: People Need Estimates. You might want to read that post before or after my post, which tells one story where estimates were "needed".

We need estimates

Prioritization in our project was initially made by dividing the features into four categories: must, should, could, and won't have. Our team had basically finished all the must haves and since there was time left before the pre-defined launch date, the management wanted to prioritize the should haves.

The should have items had already been prioritized based on business value and the request for the team was very clear: the management needs estimates for all of the should have items so that they can make the final prioritization.

Or perhaps something else?

I already revealed above what the management actually needed. They needed to decide how the development team should spend the scarce time there is left before the launch. They didn't need estimates. Surely estimates is one solution for this problem, a very typical one actually.

So what's wrong with the estimates related solution? One thing is that if you don't estimate the value, estimating the cost is quite debatable. Another thing is that in this case the should have list was very long. If the management wanted the team to estimate the whole list, it would have meant that we spend several days on trying to understand what some of the items mean and how we should implement them. Or we could have just thrown the famous "educated guesses" without even knowing what we were supposed to build.

Input for management

I didn't like either of the approaches so we did something else. We had all the items on our whiteboard vertically placed in the initial priority order. We went through the list and placed each item horizontally in one of the three columns: simple, complicated or external dependencies, and very unclear or lot of uncertainty. This took about an hour of our time.

When we presented the revised list for the management, our message was the following:

• The items in the simple column are such that you could move them up in the priorities if you like since they should be rather straightforward to implement.
• The items in the complicated column contain more risk and you may want to drop their priority if you like.
• If there are some really important items in the very unclear column, we should investigate them more so that we can understand them better. (But we shouldn't investigate all of them because that would really waste our valuable and scarce time.)

Besides that we told them that if they really need some numeric values, we can use our statistics about the development lead times. They weren't interested in the numbers, the table was just fine.

Decisions made

Probably the most interesting thing happened (or did not happen) after we had presented the table: our categories did not really have an effect on priorities. The items that were initially on the top had so clear business value so that they remained on the top although they were mainly complicated ones. The simple items did not get to the top even though there were - well, simple. And the management asked us to spend time on only one very unclear item.

This tells me the message that the estimates were initially asked basically just because that's how it has always been done. On the other hand, it tells me another message too. Our management (intuitively perhaps) understands that it shouldn't focus too much on the short-term development cost but rather on the long-term business value. This is something I really like.

Do we need ROI in prioritisation?

Return on investment (ROI) is a tool used in accounting to measure investment profitability but also in software development to prioritise backlog items. The idea is simple. Let's define the value that a backlog item generates, estimate the investment cost, and then we can calculate its ROI: (value - cost) / cost. When we have these for all items in our backlog, we can sort them by ROI and build the items with highest ROI first.

Where's the value estimate?

Although the idea is simple and compelling, I've never seen it being used in practice. Or to be more precise, I've only seen half of it being used. In other words, I've been in projects where the developers or other people try to estimate the time (cost) it takes to build something. However, I have never been in a project where the value estimates have been made for the backlog items with same accuracy. So, why to bother estimating the cost, if we don't estimate the value?

Probably the reason why the value is rarely estimated, is simply because it is more difficult. It's often quite easy to say that one feature is more important than some other feature and provide convincing arguments. However, saying how much more important in terms of money is much more difficult. I guess this is because features usually are a part of something bigger and it can be really hard to track their effect on revenues even afterwards.

Simplified ROI

So how can we make proper prioritisation if we don't know the exact value that features bring? One solution for this is to simplify the ROI formula. Please consider what happens if the cost is 1, i.e. if all backlog items are approximately equally sized. The formula becomes ROI = (value - 1) / 1, which is basically same as ROI = value. This means that we don't have to know the exact numbers anymore. It's perfectly fine to say that this feature is more important than some other and make the prioritisation decision only based on that.

Obviously this works only if the development team knows how to split the tasks into small enough pieces. If your team has tasks that take 1 day and tasks that take 1 month, you cannot use the simplified formula. But if all of your tasks basically take 1 day, you can just focus on the value the tasks produce.

You may say that the formula cannot be simplified like that if tasks sometimes take half a day and sometimes 1.5 days. The latter is three times compared to the former, right? But I think that in our highly variable software development context it's ok to estimate those tasks as an average of 1 day. Although we might have an idea that something takes half a day instead of one, the experience has shown at least for me that those estimates aren't very reliable. So from the estimating point of view, using the average of one day is just fine.


PS. You may want to read my comment from the comments section. It hopefully clarifies this post somewhat.

Will we be faster if we’ll have an extra developer?

This week I was asked a really difficult question: Are we ready sooner if we'll have a new developer?

Before exploring this any further I would like to mention that the question itself tells something about the asker. Instead of assuming that of course adding a new developer will make the team faster, the asker was aware that especially in software teams this isn’t always so. There’s even a risk of the opposite.

Nevertheless, we had to make the decision because we are in the middle of the project that is running out of time. The cost of the delay is quite big and we already have used many other tools for meeting the deadline, including reducing the scope heavily. Before I tell you what our decision was, let me explore the advice I got from the precious Twitter from my precious friends.

1. Ask first the opposite question

First of all, Torbjörn Gyllebring gave this excellent advice (read the second sentence):

I think this a great starting point. If your team would be actually faster by making it smaller, then it obviously won’t be faster by having an extra developer. I’ve been in a team having 13 developers (incl. a tester) at some stage an losing some of them didn’t make us any slower. And it wasn’t about getting rid of the bad developers. For every team there is an optimal size, which isn't "the bigger, the better".

Anyway, getting back to our current project. This advice didn’t help us because we already had a small team and removing someone would probably slow us down.

2. How well is your problem understood?

Now read the first sentence of the previous tweet. It might be that the structure of the problem is such that scaling is unfeasible. This resonates really well with me since couple of weeks earlier the answer would have been definitely "No". Our biggest challenge in the beginning of the project was to understand what we were actually about to do and especially how we should do it. The domain in a high level is quite easy to understand but the details are not. Besides we are heavily dependent on underlying systems, and the integration flows aren’t really obvious.

However, when the question was asked, we already had solved the biggest problems in this area meaning that we knew what kind of code to write. So neither this comment helped giving the answer.

3. What is your current codebase like?

The next good advice was related to the quality of the codebase:

I think that this too is a valuable advice because the quality of the codebase really affects how fast the new team member can start being productive. A crappy codebase can mean that the developer needs months in order to become productive. In a good codebase that can happen within days.

Luckily our codebase is really young. Or to be precise, we are extending an existing codebase but the features we create are completely new and I would say that at least 95% of our code doesn’t touch the existing one. So neither this prevented us taking an extra developer.

4. How good is the potential developer?

One of the advice I got on Twitter was related to the skills of the new developer. It is actually pretty obvious that the skills matter a lot. If the new developer just started coding last year, she probably won’t be very valuable if the other team members are much more experienced. In such a case the team probably requires certain level of quality and guarding that may take the precious time of the experienced developers.

But even though you’d be offered a senior developer, how could you really know her skills? We can read her fancy CV but due to the critical deadline there’s not too much time to spend on interviewing or testing the developer’s skills. What helps in this case is really simple. If you know the person beforehand, you don’t have to spend your time on that. And now we are very close to our decision…

5. Can you have a developer that’s already familiar with the system?

One of the developers in our team suggested that we could borrow a developer from another team close to our project. I really liked the idea because besides we knew his skills, we knew that we don’t have to spend too much time on teaching him. And that was our solution: we’ll be faster if we can have this particular person. An hour later we were told that he will start the next day.

Formula for selecting the answer

If we collect all the advice together, I think we have a decent formula for defining whether an extra developer would make the team faster:

1. Make the sanity check first - would removing one developer make you faster? If yes, then forget the new one.

2. Is your problem understood well enough? If not, don’t try to solve a wrong problem by getting a new developer.

3. What is your current code base like? If you don’t have too much technical debt, you might be lucky.

4. How good is the potential developer? It also helps if you know the person beforehand.

5. Can you have a developer that’s already familiar with the system? If you can, your chances increase.

I think it’s important to notice that all of these questions are somehow related to time. The questions 3-5 in such a way that you need to have more time in order the new developer to make the team faster. The questions 1-2 so that if you first solve some other problems, maybe later you can have the benefits of an extra developer.

What if I still cannot answer the question?

It’s easy to admit that the formula above isn’t very exact. Maybe I should rather use the word guidelines. Anyway, it might be that your problem domain is probably clear enough but you are not sure. Or that the developer probably is good enough but you are not sure whether she’s good enough taking account the time you have. And so on. In that case assuming that your cost of delay is big enough - meaning that wasting money for an extra developer isn’t issue as long as she won’t make the team slower - you probably just have to do what Vasco Duarte suggested:

There are things that you can only find out by trying. On the other hand, you may want to mitigate your risks. If the lost money isn’t the biggest issue, the biggest risk probably is that your team will get slower. This could be caused that the other (fast) team members need to spend time on helping the new developer. One possible solution for this is to throw the new developer into the cold water. If she can swim alone, she probably will be helpful for the team.

How about your team?

So here are my questions for you. If you were asked the same question, what would you answer and why? Would you have an extra developer? Would you make the decision based on the advice above or use some other ways to decide?

PS. I started this blog couple of months ago. Thanks for all the readers. Next year I will continue with the same goal as this year: one blog post per month. Merry Christmas and Happy New Year!

Order of programming tasks

Yesterday I had an interesting debate with my colleague while we were pair programming. We were implementing a client for REST API and argued whether we should first implement the happy case or failure scenarios. I voted for the former, he was for the latter.

My colleague was worried that if we don't start with the failure scenarios, there will be a temptation for not doing them at all. I replied that sure it requires discipline from us that we won't stop after the happy case and rush to some other programming tasks. Nevertheless, I saw more value in starting with the happy scenario because that way we could help a third developer who was implementing the orchestration part of the feature and was dependent on our code.

Later I started to think more about this conversation because I felt I couldn't argue my opinions well enough, although we decided to start as I suggested. I then realised that my argument was based on small batch and options thinking. Let's put it this way. If we had started with the failure scenarios, we wouldn't have had anything to share for the third developer before we finish both the failure and happy scenarios. I would call that as a big batch approach. On the other hand, when we did vice versa, it created the team an option to use the finished code and utilise it in the other parts of the system - while knowing that it is not the perfect code yet. This was a small batch approach.

Another kind of theoretical way to think it is from agile software development point of view. In agile software development you should be doing the most important thing next. What is most important depends on what brings most value. In our case the happy case created more value than the failure scenarios.

I still agree with my colleague that there can be a temptation to stop with the happy case and kind of forget doing the boring failure handling. But like I said above, it requires discipline, which is one attribute that can be attached to experienced programmers - who I think we are. One way to be disciplined is to write failure case tasks for the user story, another is to have a proper definition of done for stories in general. But all in all, we shouldn't give up from value just because we are afraid of not being professional enough developers.

NoSQL and #NoEstimates

Vasco Duarte asked #NoEstimatesQuestions on Twitter some time ago and collected them to his blog (part I, part II). One of the questions I asked was: Is there some analogy between SQL/NoSQL and Estimates/#NoEstimates? Vasco gave a short reply and promised to get back to the subject in a later post.

I forgot the whole question but remembered it again last week when Antti Sulanto wrote a post where he tells why he doesn't like #NoEstimates. Which reminded me of Ron Jeffries' excellent post about NoEstimates movement. What's similar for both of the posts is that they seem to be a bit worried about the extreme tone of the word No.

I see it a bit differently. I see it somewhat in the same way as the No in NoSQL. All of you who have read about NoSQL are aware how for example Twitter cannot and doesn't have to rely on traditional transactional and relational storage solutions when taking account the huge amount of data they are storing. So NoSQL doesn't mean that you should never use relational databases and SQL. NoSQL just offers superior solutions for certain kind of situations.

In the same way I'm interested to understand better what is software development like when we don't estimate. I'm not looking for software development world where we don't ever estimate in any situation.

But! If I see a case that would be typically solved with estimates, I don't want to start with accepting that as an only or best option. I rather try to understand what could be a better way. Because I am one of those who have seen how estimates have wasted a lot of valuable time without adding much value and how estimates have been used to make decisions when something else was rather needed. Neither I want to say that #NoEstimates is a stupid hashtag because there will always be cases where estimates provide value.

My personal work history contains four projects where our development team didn't estimate the work beforehand but rather focused on the flow and possibly measured the actual throughput and/or lead time. Experiences from those projects encourage me to continue exploring the alternatives to estimates. Without worrying too much about the word No.

How confident are you with your estimates?

I know you are a busy reader but before going any further I would like you to answer to ten questions. The questions are taken from the book How to Measure Anything: Finding the Value of "Intangibles" in Business by Douglas W. Hubbard. You should be 90% confident with your answers. This means that if you do the test perfectly, you should get 9 correct answers and 1 wrong.

Do the test - answer to the ten questions.

How did it go?  (I appreciate if you write your result to the comments.) When I did the test myself, I got 6 correct answers. When a couple of my colleagues did it, they got 2-5 correct. Two weeks ago I was in ALE 2013 unconference in Bucharest and held an open session there. I asked the questions from a dozen of people. Most of them got 2-4, some of them 5-6 and there was only one person with 7 correct answers.

Too difficult questions?

When people get rather low results from the test, they typically react by saying that the questions were too difficult. This is the key point here. If you don't have enough information, you should not give too narrow range. When you get more information, only then you should make the range narrower.

So how about software development? Have you ever been in a situation where your boss asks you to give a quick "educated guess" about a new feature because there is a management meeting tomorrow where the estimate is needed? And although you don't feel very confident, you say "well, maybe 30 days". And then the boss takes your (educated?) guess with him, goes to the meeting, and let's the guess to become a fact that is used to make an important business decision.

I think this is exactly such a situation where you don't have enough information to give an estimate like that. You are after all giving a range of 30-30, i.e. a single point! Instead, if you are a calibrated person (1), you could say: "with 90% confidence, I think it is 10-300 days". Or if you are not, like me, you should probably just say: "with this amount of information, I don't know".

Unprofessional answers?

When I gave that advice in the ALE13 open session, two people asked me that wouldn't it be unprofessional to give an answer like that? I replied to them that it would be actually quite the opposite.

I think it is unprofessional to pretend that you have information when you don't. I think it is actually unethical. It is much more professional to be honest and say: "I don't know".

How to narrow the range?

If you have a smart boss, he probably wants to know how to make the initial range narrower. The first thing is to spend a bit more time to think about the problem you are about to solve. You can probably identify a couple of parts that are especially uncertain. Maybe you can write a prototype or a technical spike for them?

Another approach is to ask what is the most important thing we need to solve? If your initial estimate for the whole thing was 10-300 days, maybe your calibrated guess for the most important part is 2-20 days. It is still a wide range but perhaps small enough so that you can just do it and see how it goes. And most importantly, to learn by doing.

You will actually learn many things. You may learn technically. You will understand the domain better. And you can start measuring your progress so that you can stop guessing and start forecasting.

Or you may learn that you need to build something else that you were actually supposed to build. If that happens, what is the value of estimating the whole project beforehand then? Another important aspect is that the smaller slices of work you can create, the less need you have for estimating their size.

Cost and value

If we go back to the initial question about the project size, it may actually be the wrong question for another reason as well. When we want to figure out whether to start a new project, we tend to focus on estimating the project cost because that is the "easy" part. We don't try to estimate the value because that would be too difficult.

In his book Douglas Hubbard criticizes such a behaviour. As the book title says, he claims that anything can be measured, so also the value of a project. For doing it he provides you with many tools. I really recommend you to read the book and find out more.

PS. In Finland they are planning to build a health record system that would cost 1.2-1.8 billion Euros. I wonder if that estimate has a 90% confidence interval..?


(1) Calibrated person is such that gets regularly 9/10 right when she is asked to answer with 90% confidence interval. When Douglas Hubbard has a measuring challenge, he trains the key people so that they become calibrated. One tool for that is to answer to other similar questions. After the key people are calibrated, they can give reliable initial estimates for the questions that are created based on the measuring challenge. Based on the initial estimates Hubbard defines with the aid of statistical tools what part of the challenge should be measured more in order to provide most valuable additional information with the least effort.

Marbles and batch size

Last weekend our family visited the model railway museum in Kouvola, Finland. It is a very fascinating place with all the small trains travelling in a miniature village. However, from a certain point of view the most interesting thing in the museum is a marble race game. My sons played with it and tried different ways to add new marbles. After I had watched that for a while I thought that we should record an experiment. Here's a short video I made about it:



So why share this? Well, I guess all of you who are familiar with things like batch size, flow, and kanban know what I mean. For those of you who aren't here are a couple of questions you could ask yourself:

1. What are the marbles in your organization?
2. Do you know what your batch size is?
3. Are the marbles in your organization rolling or do they get stuck too often? What might be the reason for that?