Saturday, August 24, 2013

From hour estimates gradually to #NoEstimates

This is my first blog post, welcome! I would like to share with you my real-life experience related to #NoEstimates. I've been following the discussion on Twitter this year but learned most of what I know by reading blogs. So maybe you can learn something from my post as well. The post tells a story about a team that was using plenty of time on estimation but gradually moved towards #NoEstimates. Let's see how that happened.

Initial estimation method: hour estimates


The development team I joined as a Scrum Master had a long history. However, during the previous year about half of the team had changed. I was told how it at least used to be a great Scrum team but when talking with anyone in the team, it was clear that they were struggling. I don't know if it had been like that a year or two earlier but at least at that time they were far from well-performing.

During the first sprint with them I just observed how they worked. Among many other things I noticed:

  • The sprint planning meeting was neither very efficient nor effective. The group of 12 people was divided into two teams and both teams spent typically 5-7 hours on sprint planning.
  • During the sprint planning meeting the Scrum Masters were using an electronic tool that contained user stories with initial story point estimates. The team was discussing the stories and the Scrum Master wrote tasks based on the discussion. For each task the team provided an hour estimate using the planning poker method.
  • The atmosphere during the meetings was, how should I put it, not very energized. They weren't events where the team would eagerly try to find the best possible solution to the problem at hand. I noticed e.g. how some people were so bored and frustrated that every once in a while they would just ignor the discussion and spent time on Facebook or similar.
  • One of the Scrum Masters' tasks was to print out the user stories and the tasks from the electronic tool. During the sprint the developers of course noticed new things to do but you couldn't see that on the Scrum boards since nobody wanted to write new tasks into the tool and print them out. It was thus difficult to follow the actual progress during the sprint.
  • Even though the planning was very detailed, the teams weren't able to finish the user stories during the sprint. The other team finished half of the stories completely while the other one finished none. The teams created burndown charts based on tasks and their hour estimates. This meant that if they had 80% of the original tasks done, they had a pretty “successful” sprint. It didn't matter if the initial tasks were irrelevant or if none of the stories were completely done.
The first sprint ended with a retrospective where many of the team members pointed out the problems I listed above. The team decided to try out something new.

Transition from hour estimates to story points


The next sprint planning was quite different from the previous ones. We stopped doing hour estimates. We threw away the electronic tool. We stepped away from the pressing meeting rooms and used the team space instead. We didn't try to do all of the work with the whole team but instead did some of it in groups of 2-3 people. And although we had printed the user stories, we wrote the tasks by hand.

First we checked the product backlog and picked the top four user stories and discussed them briefly all together. Then we split the team into four small groups and each group was responsible for providing the tasks for the story. As a detail I remember how someone suggested that we should write a couple of tasks together so that everyone would see what it is like to write them, how to pick them from the discussion. This was an interesting detail since I realized afterwards how the “Scrum Master uses the tool” approach had made them passive also in this sense. After 15 minutes or so we gathered together and each group explained what they had done. Others made comments and asked some questions. Based on these the team fine-tuned the tasks.

The same was repeated until finally we had about ten stories planned. The only thing we were missing were the estimates. I asked the team which one of the stories is the smallest. It was easy to find and that story got one story point. Then I took a random story and asked if it was the same size and if not, how many times bigger. That way we got story point estimates for each of the stories.

The team had been using story points also before but they were based on hours with some formula that I don't recall. Since we now had a new meaning for one story point, we didn't have comparable data from the previous sprints. Instead I asked the team: do you think that you can completely finish all the stories during the sprint? Although they were not very confident, they decided to commit to all of them. So were we done. We had spent about three hours, went for lunch, and started writing some code.

Story points era


One of the changes we made was that we stopped drawing burndown charts based on tasks. Instead, we used completely finished stories. Below you can see how it looked in the new sprint #1.



This was something I had witnessed before. It goes like this: In the beginning everybody can choose what they start to work on. Since it is the most efficient way (right?), almost everyone picks a story of their own. In the middle of the sprint none of the stories are completely done. At the end of the sprint magic may or may not happen. In this particular case they got pretty close but from an earlier team I remember how there were five developers, five user stories, all of the stories work in progress, and only one of them completely finished on the last day of the sprint.

So during the first sprints we had a lot more to improve than just make the sprint planning more effective and efficient. One thing was to start working more in pairs or small groups. Another important thing was that the developers tried to get something for the tester sooner instead of waiting for the whole story to be coded. This way the user stories were ready sooner. It also made the tester happier since he didn't have to wait until the end of the sprint to get something new for testing.

However, that wasn't enough. The team wasn't able to reach their goal during the first couple of sprints. At the end of one sprint planning one of the team members asked how many points the team had completed in the previous sprint. I said about 30. Then he asked from the team: If we have managed to do 30, why should we commit to 40 again? A good question, I would say. So they decided to drop a couple of stories away.

Little by little the team learned to commit to a reasonable amount of work and also get the work completely done in the sprint. After 2-3 months the charts started to look like this (we changed from burndown to burnup at some stage):


An important thing that the team learned was that if they commit to stories that are too big, there is a high risk that they won't be able to finish them. The team created a rule that if a user story is estimated to be more than five points, they have to split it into smaller pieces. I believe this was a crucial lesson towards the next step.

S/M/L estimating


The duration of a typical planning session had dropped from 5-7 hours to 2 hours or even less. The team was able to finish the sprint goal almost every time. But I still felt that we could do even better.

I remember that sometimes we were using too much energy on discussing if a story was one or two points. I even remember a case when time was spent arguing whether a story was zero or one points.

We also discussed if it made sense to estimate bugs and include finished bugs in the burnup chart. It felt like cheating: what if you finish a 3-point story in sprint n, find three bugs in sprint n+1, and fix 1+1+1 points in sprint n+2? From the commitment perspective (how much we'll be able to do) it made sense but from the value perspective it didn't.

There were also situations that we couldn't know beforehand whether we were able to start working on a certain story since it was blocked by an external party. Or we didn't know exactly what we needed to do since we first needed to find that out by doing another story. However, since those were important tasks that should be done if possible, we reserved space for them in the sprint backlog: “These are the stories we have selected and besides them we have 3 points for these unknown stories.”


Since all of that felt kind of like waste, I proposed the next step for the team. Let's drop the story points and instead use sizes S, M, and L. S means 1-3 old points, M means 5, and L is bigger than that. If a story was S, it required no further discussion about its size. If it was M, it was a warning that further discussion might be needed - can we really complete the story or could we perhaps split it? If it was L, we had to split it. The sprint commitment was made based on the gut feeling using the question: from 1 to 5, how confident are you that we will be able to complete all the stories we have chosen?

An interesting thing was that we never actually used those sizes. The team had learned to split stories so small that all of them were of size S. At that time our typical process was such that we had enough stories on the whiteboard waiting for the next sprint. We spent 10 minutes on them on the last day of the sprint. We started the next sprint with about an hour-long sprint planning meeting where we made sure that the whole team knew what we were going to do and checked if there was something important that was missing from the backlog. The developers wrote the tasks when they picked a story and rewrote them whenever needed. It felt like we were getting closer and closer to a nice flow.

#NoEstimates


At some stage we decided to split the team into two. The reason for this was that even though there was one code base, there were two clearly distinct businesses using it. This caused a major challenge of how to prioritize stories. So one component team became two feature teams, each business having its own.


The team I was in decided to take the next step towards #NoEstimates, although at that time I hadn't heard about such. We decided not to have sprints anymore but instead every time choose the next most important thing. Of course this meant that we tried to keep the amount of work in progress as low as possible, although we didn't have explicit WIP limits written on our board. It was important to have as small stories as possible but we didn't spend any time on estimating them (well, intuitively perhaps). We were just thinking if this story made sense and should and could we split it. Sometimes we noticed during the development that it made sense to split the story and then we just wrote a new story.


Instead of sprint plannings we started to have weekly meetings having all the relevant people from this business area in the company. That included of course the development team and the so called business people. We didn't have a Product Owner anymore since there was no need for such. In the weekly meetings we as a group talked about the big picture, checked what was going on, and decided together what we should do next. We used another whiteboard that was scaled to an upper level than what the development team had.

Instead of calculating velocity based on story points we started to count finished stories per week. Below you can see how our throughput statistics looked during the first 20 weeks. Notice especially the last eleven weeks: every week 2 or 3 finished stories. When the throughput is so stable, why would you need any size estimates?

 
It was a week 20 or so when we realized that we needed to do a major refactoring in order to meet a certain important business need. It was the first time in this new team when we needed to do estimation of some kind. Our approach was the following: Try to understand what needs to be done. Split the work into user stories or similar. Count the stories. Use the statistics to forecast what the probabilities to have this done before date X are or when all of the stories would be done with a decent certainty.

We were a bit skeptical about how the business owner would deal with our non-traditional approach of forecasting when the project would be ready and in production instead of estimating in man-days. Luckily we were fortunate to work with a smart guy and after asking a couple of questions he just said: ok, go for it.

What really happened was that the required changes were in production pretty much when we expected them to be. However, we didn't finish all of the dozen stories we had planned initially. Instead we realized that half of them could be done later and replaced those with other, more important tasks. The throughput was as expected but the content was something different, more valuable.

#estwaste and euros


Before the #NoEstimates hashtag I remember that at least Vasco Duarte was using #estwaste in his tweets. I like the word waste since it is an easy word to throw out on many occasions but let me provide you with some numbers that should make the word more concrete in this case.

If you read the whole story, you noticed that we started with sprint planning sessions that lasted about 6 hours and in the end we didn't have them at all. If we assume that there are 22 sprints per year and the team has an average of ten members, it means 1320 saved hours per year. I really don't know what the average hourly cost of the team members was but let's pick two numbers: 50 or 100 EUR/hour. On a yearly level this means savings of 66,000 or 132,000 euros. Besides that you probably noticed that we didn't need the Product Owner anymore. So you can add the cost of one manager above that.


I guess you are now saying that I forgot the value part of those sprint plannings or that I forgot the cost of the one-hour weekly meeting. Well, I didn't. First of all, the old sprint plannings produced very little or even negative value. Surely the developers discussed the upcoming work there but I would say that the discussion wasn't very useful. One of the purposes of the plannings was to provide visibility for the Product Owner but it was hard to see such an effect. And the usage of technical tool caused problems during the sprints since the team was having difficulties using the new information they learned while working. With negative value I refer to the drop in people's motivation.

Instead, the weekly meetings really produced value. They helped us to share information very efficiently and make useful prioritization decisions. So the cost calculations above really refer to the waste (=no value added), although they even ignore things like opportunity cost, cost of delay, and so on.

Lessons learned


Let me choose the two most important #NoEstimates lessons that I learned during this journey. The first one is that at least in this kind of context the #NoEstimates approach is perfectly valid and can bring huge improvements for the organization. With “this kind of context” I mean an ongoing product development. Unfortunately I don't have experience on making business decisions before starting to develop a large-scale product. I would love to read your post about that topic.

The second one is that if you start from the situation described above, you cannot just jump to #NoEstimates. Instead, you have to find your own path and take small and sometimes bigger steps towards it. Vasco Duarte claims that story points are harmful. I understand what he means but that statement depends on the context as well. In this post I described how we gradually moved from hour estimates to story points, to S/M/L sizes, and finally to #NoEstimates. The story points helped the team to learn how big stories cause problems and split the stories into smaller ones. I feel it was a necessary step to take.


I think that working without estimates requires that the team has a certain maturity level. If your team doesn't have that yet, you need to work hard (smart) in order to get there and enjoy the benefits of #NoEstimates. That is what we did and I recommend it for you as well.


5 comments:

  1. Good story. I see many similarities in your case to what I have seen in my work and with my customers regarding the progression from 2-day reviews+plannings to spending just an hour or so to do the actual plan how to proceed with new items. Cost reductions have been quite significant and speed of delivering #TheRightStuff has increased. What I find interesting in your case was the absence of PO. I assume you had proper system in place to get business feedback and guidance to the team or then you worked closely with the customer so that PO role would be redundant.

    Doing #NoEstimates when you have established team and established, ongoing product to be developed further is a no-brainer. Instinctively it might be a tough sell for completely new development or even in situations where existing product would be used as a basis for new developments. We did just that in our last big product development effort with quite a bit of success. Why? Because we had a deadline to deliver and not much time to deliver so we decided not spend time on estimation but spent much more time defining the ambition levels per feature and understanding the absolute minimal set of features that we could ship with. Probably I should post few blog entries about it.

    ReplyDelete
  2. We had a close relationship with the business people. One of them could have taken the PO role but that wasn't necessary. Like I wrote in the blog, we had the weekly meetings and besides that we invited the business people to our daily standups. The priorities were decided together based on discussion instead of everyone looking at one person and asking what to do next. This worked very well and made the PO role redundant.

    ReplyDelete
  3. Recognisable story. We started about three years ago with agile development of software for our naval command and control systems. Lead by the instructions of Cohn, Kniberg, Sutherland et.al. we have been able to establish a solid agile development organisation of 5 to 8 scrum teams.

    However we have also experienced the journey from strict scrum and estimation practices to more relaxed ways of using stories, tasks, hours, storypoints etc. As pointed out, it requires mature teams to reach such a "next level". Not all teams are yet able to jump-start this approach.

    We have not reached your #noestimates continuous flow maturity level. But I expect that some of our teams might be able to reach it in the foreseeable future as well.

    Very useful 1st post!


    ReplyDelete
  4. I'm sure many teams would want to go your way, but there's one aspect that I feel you didn't touch: how to get the business people to play along with you? Say your team has a PO: how will that person like it when the team starts discussing the matters past him? Also, in your experience, how did the business people like it when your team started to steal their time, maybe an hour a sprint or that 10 to 15 minutes each day? Did anyone try to cheat there, or were they all just smart enough to see it from the beginning that this is the right way to do it?

    ReplyDelete
    Replies
    1. We didn't start to discuss anything past PO. Most of journey the PO was there just like in any other Scrum team. The time without PO started only at the end when we had formed a close relationship with the business people. At that time there was no need for a PO any more, at least in such a role that it used to be before.

      And we couldn't have had a good relationship with the business people if they would have been forced to some meetings. Instead they quite quickly realized how valuable it is for them to gather once a week around the board to discuss what has happened and what is going to happen next. The meeting was not organized only because the development team needed it. It was organized because it was an excellent tool for the business people to manage the process. And all the people involved said it was one hour well spent.

      The daily 10-15 minutes with the business people were extra above this. They were useful because we could make the daily prioritization decisions (when needed) and share such information that would have otherwise been shared later or not at all. The same way as any Scrum team sees daily scrum valuable.

      Delete