Wide-Band Delphi Estimation for Business Value – 2
See Part 1.
Step by step
Let’s walk through how to do it.
- We have a Product Backlog, usually user stories.
- Identify the right people to be the Business Value experts. We recommend they have participated in developing the PB. Typically five people. I call them the Business Stakeholders and the Product Owner. The five best experts. (Sometimes you have four or six people.)
- We discuss the BV drivers to make the concept of ‘Business Value’ more concrete in this specific situation.
- We choose the highest BV card. We give it 100 BVPs (BV points). It becomes the reference story for BV.
- We select stories randomly and compare them to the reference story. The experts vote on BV using the Fibonacci cards. I recommend the full true Fibonacci series up to 144. If you want to add some cards at the high end (some, not many), I think it will not hurt.
- Discuss and then ‘secretly’ vote. 1-2-3 show. Everyone shows his poker card. If the cards are all within three Fibonacci numbers of each other (e.g., 21-34-55), sum the cards and average to the nearest integer.
- When the vote, they are voting comparative Business Value, or relative value.
- If they are not close, then discuss (especially the highest and lowest voters), so the five voters learn something, then re-vote.
- If the average of the votes is 50 BVP, then the experts believe the new story has 50% of the BV of the reference story (100 BVP).
- Allow about one minute per story, on average across all the stories. Maybe slightly more. They will discuss the first few cards more slowly, and the last half of the story cards will go more quickly. Do not let them take more than 75 minutes for 50 user stories — go fairly quickly.
- When you are done as a team, arrange the cards of similar BVPs. Ask the experts: Do you see any cards that look ‘out of kilter?’ Often they will identify a few user stories to be re-voted.
That’s the basics. Pretty simple.
I have never seen any set of experts vote all or even most of the user stories as 100 BVP (on every one). In fact, their votes have roughly an even distribution from 100 to about five or two BVPs across the set of stories. This is quite remarkable when you think of what we used to do.
Wow. I recommend that the BV experts think mainly in terms of ‘wow factor.’ That is, BV is what will make someone (e.g., the customers) say ‘wow.’ Or say wow more loudly. The louder the wow, the higher the number.
Dependencies. The experts often talk about dependencies (of one type or another) and want to deduce that the one card on which many other stories are dependent — that card must therefore have the highest BV — I discourage that way of thinking. I recommend handling dependency issues a different way.
PO Role. If they need a decision about what a story is (or is not), the PO is the final decision-maker. We hope, of course, that the PO uses common sense.
ScrumMaster. We recommend you use a ScrumMaster (or someone like that) to facilitate the meeting. Keep them at a good pace. Let the quiet ones talk more, get the talkative ones a bit less verbose.
Implementers. I like to let the implementers listen. They learn things, and I like to let the implementers, occasionally, ask questions.
“Why is that story so important?”
“I thought that story would be more important… Can you explain?”
The experts take a minute or two to answer, but we cannot distract the experts too much.
Popcorn. The voting is very interesting to watch. Bring popcorn. You could even sell tickets. The experts will disagree with each other, educate each other and learn things about Business Value. They will complain that it is hard to estimate the future. The implementers usually find it amusing when the business people say that, because these are the same ‘jerks’ who two weeks ago ‘beat up’ the implementers for not being able to estimate effort more accurately. They start to respect each other more.
Average. Average the votes of the experts. This means that the stubborn person does not get to dominate. This means that everyone’s opinion is counted, so people participate more fully.
Re-voting later. Business Value is hard and very important. They will complain. Some may say that a few cards are ‘off,’ but remind them that that was the average opinion of all the experts. And, I strongly recommend, tell them that we can revise the numbers any time we are smarter. When we have more numbers, after we have done a demo, after some useful feedback from customers. Then, with that new information, we can re-vote a few of the stories. Saying this allows some people to ‘give up on perfect estimates’ sometimes.
Re-voting now. Notice in Step #11, if we notice any stories that are ‘wrong,’ we can re-vote them now, but the group has to agree the re-vote is reasonable, and it should only be a few cards that are re-voted.
Motivation. The implementers learn a lot about Business Value, as they watch. They typically get a much better sense of the purpose of the work. This changes their motivation and it will change their behavior when they build the product.
Experts. You may learn that one or two of your ‘experts’ are not very good. You may need to act on that information. You may learn that a key expert was missing.
The unbearable lightness of estimation. Are the experts really expert? In my opinion, we can say that they are the best people we can find to make these guesses, but they are really guessing what the customers will want when we deliver the product and for the two to five years after that. No one is really an expert about the future.
Guess and act! Still, business has always been risky and the future uncertain. We get the best people to take their best guesses, and we roll with that. We hope that will give us more success than our competition. We expect to learn more from action. For example, we fully expect some re-voting.