Using Velocity to compare 3 Teams

Yesterday I gave a webinar on “Velocity and Story Points”.  The deck is on this website.

Pooya asked a longer question, but I’ll start with this simpler question:

“Can you use Velocity to compare 3 Teams to each other?”

Key Preliminary

Scrum is a game.

If you do it right, people like having things “gamified.”  A bunch of benefits; beyond the scope of this short post.

Some Basics

Let’s make some easy observations first.

The first principle is, commonly, you want, as we say, an apples to apples comparison.

But notice this.  To be fair you want the following:

• All 3 teams do very similar work
• All 3 teams use the same reference story or stories
• All 3 teams have the same number of people (not the most important factor)
• All 3 teams have very similar people (eg, roughly same skill level)
• All 3 teams have people with the same level of motivation (motivation is very important for knowledge worker productivity)
• All 3 teams have similar impediments
• All 3 teams are equally skilled in estimating (I assume using Planning Poker)

It is rare, I think, when all these conditions (and probably other similar ones) apply.

BUT, if that list were all true, then you make a comparison, and we see a difference, and probably we can act on it quickly. And get some benefit.

Worth noting: I think most knowledge workers would be de-motivated by unfair comparisons.  Even those teams who look better.  Be careful here.

Short Obvious Comment

My mother would said: Comparison are odious.

When making comparisons, we assume no one is being odious. Obviously.

What is the Goal or Purpose of the comparison?

Why are you wanting to make the comparison?

I already posited one reason:  The manager wants to identify something useful to fix or improve on.

A good goal.

I suggest each Team create an Impediment List of 20 things to improve on, in priority order.  That would be a better place to start.

In part, as I have suggested, because there is so much “noise” in the signal of Velocity, when you compare Teams.

Another use: To increase the pay of Team A because their Velocity is higher than Team B.

Hmm.  That raises many questions about how to review performance and how to adjust compensation over time.  A VERY difficult subject.

My first observation would be that compensation should be used, of you can, to optimize the most important outcome you want.  In my opinion, that outcome is often better described as business value delivered to the customer.  Yes, productivity is a factor, but only one. (Velocity is supposed to show improved productivity.)

But I offer this quote (a bit simplistic):  “It is more important to deliver the right thing, rather than the wrong thing fast.”

Let’s rephrase with an example: “Probably better to deliver $1 million in value in 3 months, than deliver $500,000 of value in 2 months.”  And let’s assume both deliveries were the same amount of “work”.

Do not reward working hard (or productivity) if you really want BV delivered.  You tend to get what you measure.

What if you have 3 Teams on one (big) Product Delivery?

We can’t cover everything on this topic.  But let’s consider one scenario.

You have 3 teams.  Yes, if all 3 teams were equally productive and each team had a “complete” skill set, it would be easier for you (eg, as the Chief Product Owner) to manage this group, at least at the high level.  For example, easier to forecast when the next Release will be completed.

But, I think these assumptions probably do not hold for you.

So, what can we start to do?

First, pick 6 Developers (2 from each team), and have them estimate the Master Product Backlog.  The PBIs, once they are assigned to a specific team, will have to be (re)estimated by that specific team.  But as a first pass, let’s use this other set of “Developers”.

Let’s say your 3 Teams have been working for a while.

Team A Velocity is 20 SP

Team B is 25 SP

Team C is 30 SP.   And the total across all 3 teams is 75 SP.

The next thing is a rough comparison of “real history” against our “fluffy” SPs on the MPB.

So, do you think that the 75 SP total of work just produced by the 3 Teams is fairly close to the next 75 SPs (estimated a somewhat different way) on the Master Product Backlog?  In simple terms, there are 3 possible answers (I hope): about the same, a little more (eg, = 80 SP on the MPB) or a little less (eg, = 70 SP on the MPB).

Now I think you have what you (should) need:  a rough idea how long to get what you want.  OK, let’s work that out.

Let’s say you need to get another 750SP from the MPB done for the next Release.  Let’s say the answer was “same”.  So, then to get that 750SP done will take roughly another 10 Sprints (of all 3 teams).

This initial rough estimate ignores some possibly very important issues.

One example: Let’s say half the remaining work is best done by Team A, and Team B could do that kind of work, but more slowly.  And Team C will be fairly helpless doing that work. (And this issue was not reflected in the most recent sprint completed.)  Uh Oh!  So, you must adjust for risks like that.

***

Hope this discussion will be helpful to you and the people and situation you are dealing with!

« « The New Methodology || Breaking Down User Stories – some resources » »