Wide-Band Delphi Estimation for Business Value – 2

Revised: 9/26/2023.

See the first post on this topic: Part 1.  At the bottom is a link to the third post.

---

Step by step

Let’s walk through how to do it.

1. We have a Product Backlog, usually user stories.
2. Identify the right people to be the Business Value experts. We recommend they have participated in developing the PB. Typically five people. I call them the Business Stakeholders and the Product Owner. The five best experts. (Sometimes you have four or six people.)
3. We discuss the BV drivers to make the concept of ‘Business Value’ more concrete in this specific situation.
4. We choose the highest BV card. We give it 100 BVPs (BV points). It becomes the reference story for BV.
5. We select a story randomly and compare it to the reference story. The experts vote on BV using the Fibonacci cards. I recommend the full true Fibonacci series up to 144. If you want to add some cards at the high end (some, not many), I think it will not hurt.
6. They discuss. Ask questions. “What is the story, really?”  You can take notes. Sometimes they make assumptions — these need to be checked later.
7. Then each person ‘secretly’ votes. 1-2-3 show. Everyone shows his poker card. If the cards are all within three Fibonacci numbers of each other (e.g., 21-34-55), sum the cards and average to the nearest integer.
8. When they vote, they are voting comparative Business Value, or relative value.
9. If they are not close (votes are wider than 3 consecutive Fibonacci numbers), then explain (especially the highest and lowest voters), so the five voters learn something, then re-vote.  They give facts, insights, ways of thinking.
10. If the average of the votes is 50 BVP, then that means the experts believe (on average) that the new story has 50% of the BV of the reference story (100 BVP).
11. Allow about one minute per story, on average across all the stories. Maybe slightly more. They will discuss the first few cards more slowly, and the last half of the story cards will go more quickly. Do not let them take more than 75 minutes for 50 user stories — go fairly quickly.
12. When you are done as a team, arrange the cards of similar BVPs. Ask the experts: Do you see any cards that look ‘out of kilter?’ Often they will identify a few user stories to be re-voted.

That’s the basics.  Pretty simple. A 12-Step program.

Results

The results are always useful.  We get the average of what the experts think.  That’s the best we know today.

And we can use that to build on.  To get smarter, and to decide what to do first.

But, you say:  “When I have asked them to rank the requirements, if they have 100 items, they have ranked 98 of them as High (in a High, Medium, Low scale).  Won’t they do the same with this?  As in, vote most of them 100 or a very high number.”

I have never seen any set of experts vote all or even most of the user stories as 100 BVP (on every one).  In fact, their votes have roughly an even distribution from 100 to about five or two BVPs across the set of stories. This is quite remarkable when you think of what we used to do.

Comments

Fun!  This is a game (Priority Poker).  It is important that it is a game. And important that it is fun.  They will enjoy it. They will do it often. Their minds are more open and creative. And they want to learn from all the information that everyone is sharing.

Learning.  It’s all about the knowledge-sharing and the learning.  The person who voted highest explains why, or also the one lowest.  They share tacit knowledge (knowledge that is not explicit).  Very valuable.  They all learn.  And use that learning to re-vote. Cf. The Six Blindmen and the Elephant story, here.

What is Business Value?  Indeed.  How many poets for how many centuries have struggled to explain what love is? Your experts also will struggle with “what is business value.”  Interestly, they do not agree.  So, early in the voting, there will be a discussion of this.  Maybe clearly and obviously, maybe as a subtext.

Common Ideas about Business Value.  As you probably know, business people have two main theories of business value. One, it is all about the money “shareholder returns” (dividends and increased stock price).  Sometimes expressed as dollar revenues or profits.  Two, it is all about customer satisfaction.  In our context, I actually recommend another theory, Wow Factor.

Wow Factor. I recommend that the BV experts think mainly in terms of ‘wow factor.’ That is, BV is what will make someone (e.g., the customers) say ‘wow.’ Or say wow more loudly. The louder the wow, the higher the number.  And because they are so excited, they get out of the chair and (metaphorically) come to us and demand the product now.  And they do not care about the price.  Ok, you get the basic idea. Some features have low Wow Factor, some medium and some high.

Dependencies. The experts often talk about dependencies (of one type or another) and want to deduce that the one card on which many other stories are dependent — that card must therefore have the highest BV — I discourage that way of thinking. I recommend handling dependency issues a different way.

PO Role. If they need a decision about what a story is (or is not), the PO is the final decision-maker. We hope, of course, that the PO uses common sense.

ScrumMaster. We recommend you use a ScrumMaster (or someone like that) to facilitate the meeting. Keep them at a good pace. Let the quiet ones talk more, get the talkative ones a bit less verbose.

Implementers. I like to let the implementers listen. They learn things, and I like to let the implementers, occasionally, ask questions.

“Why is that story so important?”

“I thought that story would be more important… Can you explain?”

The experts take a minute or two to answer, but we cannot distract the experts too much.

Popcorn. The voting is very interesting to watch. Bring popcorn. You could even sell tickets. The experts will disagree with each other, educate each other and learn things about Business Value. They will complain that it is hard to estimate the future. The implementers commonly laugh.  They find it amusing when the business people say that, because these are the same ‘jerks’ who two weeks ago ‘beat up’ the implementers for “whining” about estimate effort more accurately. All of a sudden, sympathy breaks out.  They see they are all in the same boat, up the same creek, without the same paddle.  They start to respect each other more.  Very useful, laughter is.

Average. Average the votes of the experts. This means that the stubborn person does not get to dominate. This means that everyone’s opinion is counted, so people participate more fully.

Re-voting later.  This is essential.  Business Value is hard and very important. They will complain initially. Some may say that a few cards are ‘off,’ but remind them that that was the average opinion of all the experts at the time.

And the point was to make clear what we currently know.  To see what we can learn, and use that to improve.  I strongly recommend: tell them that we can revise the numbers any time we are smarter. When we have gathered other data, after we have done a demo, after some useful feedback from customers, etc. Then, with that new information, we can re-vote a few of the stories (perhaps each sprint).  Saying this allows some people to “give up on perfect estimates” more quickly.

Re-voting now. Notice in Step #12, if we notice any stories that are ‘wrong,’ we can re-vote them now,  but the group has to agree the re-vote is reasonable, and it should only be a few story cards that are re-voted after the first round (first day).

Motivation. The implementers learn a lot about Business Value, as they watch. They typically get a much better sense of the purpose or value of the product. This improves their motivation and it will change their behavior when they build the product.  Lower value stories are done in a simple way, higher value stories are made more robust.

Right Experts? You may learn that one or two of your ‘experts’ are not very good. You may need to act on that information. You may learn that a key expert was missing.

The unbearable lightness of estimation. Are the experts really expert? In my opinion, we can say that they are the best people we can find to make these guesses, but they are really guessing what the customers will want later: when we deliver the product and for the two to five years after that. No one is really an expert about the future.

What’s the point?  Some of us can be cynical about this.  What is the point of guessing about the Future?  Que sera sera!

The point is to see what we know, and what we don’t know.  Then prioritize our stupidity (I say that in the nicest way possible, and with a humbling smile and chuckle) and then go find new useful knowledge that will change our votes, help us increase the business value in the product. And, by the way, that new knowledge in general will help us also build the product better.

Guess and act!  Business has always been risky and the future uncertain. So, each vote is a group guess based on the knowledge at that time.  We get the best people to take their best guesses, and we roll with that.  Then we learn and re-vote as much and as often as our new knowledge tells us.  We hope that this will improve our odds of success against our competition. We expect to learn more from action. For example, we fully expect some re-voting after building some stories.  In fact, I would expect some stories are voted or re-voted every sprint.  As we identify new information, identify new stories, do demos, talk to the customers, break down stories, etc.

Emergence.  With the collective efforts of a relatively small focused team, the truth starts to emerge.  (See the emergence article in Wikipedia if emergence is a new idea to you.)  The truth is about many things: the problem, the solution, what customers want now (and in the future), the product, business value.

Enjoy!

---

Go to Part 3.

« « Wide-band delphi estimation for Business Value – 1 || Multitasking » »