Be careful with that Burn-up chart!

This blog series will explain how choice of metrics can help a team in mind shifting from resource utilization towards flow orientation.

The first part is a “lessons learned” story, illustrating why it is a bad idea to focus too much on the burn-up chart.

What we did

The story is that we gathered the team for a Root Cause Analysis. We had opportunity to talk about how to improve our low throughput. From looking at the burn-up chart, showing delivered stories each week, we could see that in the latest three weeks there were not very much delivered.

Burn up, showing weekly throughput of user stories

Burn up, showing weekly throughput of user stories

In addition we looked at a column diagram of fixed bugs. These columns hits the roof over the same iterations. This pattern occurs each time when the company enters the big integration test phase.

Visualisation of correlation between nbr of closed defects and closed user stories

Visualisation of correlation between nbr of closed defects and closed user stories

Starting it wrong with bad formulation of the problem

Me, the team coach, formulated the problem from topmost perspective. I said:

– As we see, throughput of stories has gone down. Let’s talk about this. To start with, is it a problem really?

The reply from team was

– No, I do not think there is a problem. There were some really big bugs and there were some people away these weeks. It is not strange that throughput goes down.

Then there were discussions about the graphs (that the team had not chosen themselves). The team was uncomfortable with that graphs were not showing how much effort was actually put down, and that the graphs did not show the sizes of defects. In addition, the burn up did not say anything about how many people were at work.

I had assumed that it would be clear from the column diagram that we did lots of work but with wrong things. The reason for low throughput was very clear for me, that we had almost completely worked with incidents and bugs. However, I did not hear any comment about that. Not even when I directly asked about it I got any reactions. Focus was instead on the throughput, and what was not shown in the burn-up graph. Now, given how the problem was formulated this is not very surprising, right?

What the problem formulation led us into

It was suggested that we should improve estimates on defects, so that we could visualize clearer what was done in the iteration. To manage this, we introduced shirt-sizes S, M and L. We could use the three clusters on the histogram as reference; S corresponds to cluster around 1 day, M is cluster around 6 days and L is cluster around 10.5 days.

histogram

Distribution of cycle times for defects

But wait a second here… Defects are re-work! We would be better off if they were not occuring in the first place. Instead of aiming at reducing waste (defects), we ended up with doing more wasteful work with estimates for the defects.

Let me say that again: Because we focused on throughput from the team (resource utilization), we ended up with adding non-value-adding time to our waste of re-work. #fail 🙂

Summarizing the meeting

We did not manage the RCA exercise, we lost track of the flow principles and we added more waste to our process. I closed the meeting, summarized the results and said “We need to meet again”.

Lesson learnt

I think we got on wrong track already from start, because of the visualization of the burn-up chart. Maybe, if I had instead visualized delivered stories vs incidents and bugs, the discussion could have been more driven towards removing waste.

Next meeting (and blog)

On next meeting we will bring some value-added-time graphs and put up a question formulated in a bit more guiding way:

– How can we improve value-added-time ratio?

Who lives will see…

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s