The Nokia Test

December 2, 2007

Nokia (the cell phone maker) uses Scrum. Well, actually it is/was a small Euro 50 billion joint venture called Nokia Siemens Networks. They have developed a test to check whether a team is really using Scrum or just doing what I call Cowboy Agile, or doing Agilefall (talking Agile terms, but really doing mostly waterfall).

The Nokia Test is in two parts.

First, are you doing Iterative Development?

  • Iterations must be time-boxed to less than four weeks
  • Software features must be tested and working at the end of each iteration
  • The iteration must start before specification is complete

The experience is that if you ask a lot of “Scrum” teams if they can pass this part of the test, they can’t. If you are at a conference, often not a single team in the room.

The next part of the test checks whether you are doing Scrum (in Nokia’s opinion):

  • You know who the product owner is
  • There is a product backlog prioritized by Business Value
  • The Product Backlog has estimates created by the team
  • The team generates burn-down charts and knows its Velocity
  • There are no project managers (or anyone else) disrupting the work of the team

My reaction:

I think this is an excellent way to deal with Cowboy Agile or Agilefall.

Let me say this loud and clear: A firm can’t in good faith say “we tried Scrum” and then move away from it if they never had any teams that could pass the Nokia test, and pass for a reasonable period of time. Of course they could say “we tried Scrum,” but they did not. (I will not define right now what ‘pass’ would require.)

The Nokia Test is demanding — that is clear — but a test of this nature is a necessary (if not sufficient) condition to doing professional Agile software development. (Or did we expect to get out of Fred Brooks’ tar pit by doing unprofessional software development?)

There are some forces in a firm that want to do Cowboy Agile or Agilefall or want Agile to fail (“It’s moving my cheese”). This is true in every firm that I have worked in, I believe. So, if you use the Nokia Test, expect to get some resistance.

The Nokia Test is arguably too minimal. There are many other important parts of Agile or Scrum that it does not cover. Are the things in the test the most important parts? My thought is that this collection of principles and practices provides a good core of Agile/Scrum that will defend you from the most common dysfunctions. Not all. Is there a better test? Probably we could define one, but let’s pass the Nokia Test now.

Should the test be significantly more detailed? I think not. First, to work with any test, we need the test to be simple enough to be comprehended easily by most of the people involved. In this way, it becomes self-policing. Second, Agile/Scrum needs to be adaptive, so having a lengthy test that puts all teams in the same straight-jacket would noticeably limit that adaptability.

What are your thoughts? Are you aware of similar tests? How does your firm limit Cowboy Agile?

« « Carnival of the Agilists – 11/29 || Suggested Resources for attendees at the Jim York Certified ScrumMaster course on Dec 4-5 » »

Posted in: Better Agile, Nokia Test
Tagged: Tags: , , ,

33 thoughts on “The Nokia Test”

  1. Wolfgang Schulze-Zachau

    I would see one valuable addition to the second part of the test:
    The team must hold retrospectives at the end of each iteration and the resulting “try” items must have an impact on the next iteration(s).

    Feedback is a very important aspect of all agile methods and Scrum in particular, IMHO.

    ↓
  2. Joe Little

    Hi Wolfgang,

    I have appreciated your posts elsewhere. And agree with your post.

    As a minor quibble…When I see the word “try” I always think of that Star Wars film where Yoda says, “There is no try; there is only do or not do.” So I would call them “action items” or perhaps “experiments”.

    Having said that, I think there is a big advantage to coaches to say “this is someone else’s test”. (Yes, I am an Agile Coach.) So I hesitate to modify the Nokia Test for that reason alone. Unless it became in truth the “Google Test” or something similar.

    Thanks, Joe

    ↓
  4. Joe Little

    Aha! Yes, I do that “google test” all the time, but I was not referring to that. What I meant was…if Google itself came up with a different (maybe slightly more comprehensive test for Agile/Scrum, that might be better (more useful) than the Nokia Test. If Joe Little came up with an Agile/Scrum test, I don’t think it would have the same power to influence others.

    ↓
  5. wAsh

    On the thought about a more detailed assessment and not to muddy the waters, but Mike Cohn and Kenny Rubin developed an Agile assessment survey that teams/projects/individuals could take. I thought that part of the attraction was some kind of assessment after you completed the survey. I do not have the URL sorry.

    ↓
  6. Joe Little

    Hi,

    Good thought. I have worked with a similar assessment (at least I guess it is similar), where we asked a team a bunch of questions. Good feedback for the team, and a good learning experience for them. Perhaps I will post that assessment form shortly.

    But I view the Nokia Test differently. One of its chief values is that it is not a test “I” created. It comes from a well-known and successful company. So, it has a kind of influence “my” assessment (or even Mike’s and Kenny’s) could never have. (Mind you, Mike would have a lot of influence on me, but others might say he is an Agile “insider” and “not in touch with our real-world”. Not that I would agree with those comments, but they would indicate that Mike’s influence is limited with those people.)

    Thanks, Joe

    ↓
  7. wAsh

    I totally understand about providing something that provides an independent, recognized way for the assessment. In my company, we have been developing our own detailed assessment, and also reviewing independent measures for the same reasons. 🙂

    Cheers,
    -Lyndon-

    ↓
  8. Joe Little

    Hi Lyndon,

    Aha! I for one would be very interested to hear what you all are thinking. And doing. Please share when you can.

    And perhaps you are with a firm that is well-known in its industry. But, as you say, that is not required by every audience.

    Thanks, Joe

    ↓
  10. Joe Little

    Bas Vodde. Bas also commented recently that the Test is not used as much, since everyone is already compliant. However, Nokia is a big firm, and “everyone” is probably too much to say. In my experience, people don’t conform to any standard with consistency over time. But this all does not, in my opinion, affect the rightness of the Nokia Test.

    ↓
  12. Joe Little

    Hi Martin,

    Well, not sure about the Joel Test. I can see someone having his pet peeves, but it strikes me as that, not a well-considered, consistent or proven set of considerations to quickly judge if a team is on track.

    Not sure that the Nokia Test is fully proven either. (Although Scrum is, in my book.) But it does seem coherent (the pieces stick together). And the core of iterative/scrum. And Scrum is proven, so indirectly the Nokia Test is proven. Also, the Nokia Test has “passed muster” at a great firm (or five). Not sure if the Joel Test has.

    My opinion so far. Or give me more facts or reasons to view things differently.

    Thanks.

    ↓
  13. Denni Bonn

    Hi, you’ve got an interesting blog. I like it and I like Nokia. I had several cell phones and Nokia turned out to be the best of them. Although they say it is one of the most harmful phones in terms of human health but still it is great. On this great site http://www.pissedconsumer.co I read a lot about the company and its products feedbacks.

    ↓
  16. JP

    Hi, I’m writing my bachelor essay about Scrum and is wondering if there is any references from studies or journals about the test. I’m having problems finding any… Please email me.

    ↓
  17. Joe Little

    Hi JP,

    You ask a good question…although as a scientist (amateur) I need to clarify your hypothesis.

    As to the Nokia Test itself, I think there have been a few unscientific (although probably still reasonably accurate) test of it. In the form of asking people how well do they comply with it. The answers seems to be low, on average. And, according to Jeff Sutherland, me and others, the lowest are probably getting minimal benefit; low to medium, some benefit from Scrum; high, often getting great benefits from Scrum.

    In other words: we knew this, the Nokia Test is helping a bit, but we really need a truly scientific (albeit modest) test of the Nokia Test.

    If your hypothesis is more about Scrum itself, then I can tell you that the overwhelming anecdotal evidence is that it works. Much as I said above. Here, we have many documented case studies, but we need many many more. And we need more truly scientific studies that “prove” that Scrum works.

    Go to the Yahoo group scrumdevelopment and ask your question. Also, send me an email and I can help a bit more.

    Regards, Joe

    ↓
  18. TelefoaneMobile

    You were asking at some point if this test should be more detailed. I think not because it doesn’t test HOW is Scrum doing, but rather if it IS Scrum or NOT.

    ↓
  19. Anonymous

    ThoughtWorks has a very comprehensive agile test that tells you if you are really doing agile or not.
    If you pass that test you are likely to be compliant with the Nokia Style as well 😉
    ThoughtWorks' test can be used to asses individuals, teams, an entire department or even an entire company.
    I recommend taking the test and follow up on any shortcomings. However, IMHO agile is not religion or a black/white thing. It is common sense and one is free to adapt it or one self to the environment one is working in 🙂

    ↓
  20. Paul J. Heidema

    Interesting… how do you think OpenAgile fits into the mix with Scrum, Lean, and XP ? OpenAgile is a value delivery system closely connected to other agile methods. Do you think that building capacity and progress through strengths is useful for a team using agile? Thank for the article.

    Warm regards,
    Paul Heidema
    OpenAgile Champion
    http://www.openagile.com

    ↓
  21. 5CentsWorth

    The Nokia test is good but has limitations, I coach teams, scrum masters and PMs. I dont understand why so many scrum people have an issue with PMs, the role changes but the role is still valuable. Its not a competition, its not PM or Scrum Master. Having a PM helps allot, they dont get involved in the operational and tactical running of the teams but they do look after organisational issues and dealing with external entities such as 3rd party providers. Basically the PM is 80% externally focused and the Scrum Masters 80% internally focused. Scrum Masters focus on the road ahead 2 -3 iterations, PM on the journey as a whole from here to the end. Works very well, one last thing the Nokia test implies the requirements are not fully cooked before they enter the team. i have found its the how the requirements are defined that impacts not the degree of detail. under cooked and over cooked are equally evil. Balance is needed. They need to be enough that the team believes they can deliver the requirement and there are no missing external prerequisites.
    Last of all beware all of these "tests" they are not the way the truth and the light, just a lenses to look at your working practices through, none is ever perfect.

    ↓
  23. Antique shops online

    ciao!..
    hi wolfgang, When I see the word "try" I always think of that Star Wars film where Yoda says, "There is no try; there is only do or not do." So I would call them "action items" or perhaps "experiments".

    ↓
  25. Joe Little

    Hi Amit,

    Not sure if I follow you. I agree in the sport of running, sprinting or short races are different than marathons or long races. But how are you meaning this metaphor in our context?

    Thanks,
    Joe

    ↓
  26. Anonymous

    I do prefer words like:
    “We sold software”;
    “We sold phones” ;
    “We earn money ”;
    And use Agile principals to do that in efficient way.
    I don’t care if someone you are not using all principals it is wrong, you have bad Agile.
    Do not use all principals is not bad.
    The goal of each business is to earn money.

    Blind appeals to follow point by point Agile principals – it is typical good wording blogger (b-parrot) incompetence that cannot be taken seriously.

    ↓
  27. Joe Little

    Hi Anon,

    I agree that results are important.

    But actions to get results must be based on values, principles, ideas.

    We follow values, principles, ideas to get results. So we must discuss them. But we do not follow values, principles, ideas for their own sake. It least if they are not fundamental moral principles.

    But in one blog post, one cannot explain everything.

    Thanks!
    Joe

    ↓
  30. Yoram

    Just thought I would add my blog to your site..trying to get my experience with SCRUM out there..
    cheers
    I would appreciate a refer to

    scrumdev.blogspot.com

    Yoram

    ↓
  32. telefonosandroid

    I would see one valuable addition to the second part of the test:
    The team must hold retrospectives at the end of each iteration and the resulting "try" items must have an impact on the next iteration(s).

    Feedback is a very important aspect of all agile methods and Scrum in particular, IMHO.

    ↓
  33. Joe Little

    Hi telefonosandroid,

    I don't control the ScrumButt or Nokia test. So, I can't add or subtract.

    First, I would totally agree that the ScrumButt test does not cover everything. But it was not meant to, either.

    And I do agree that good retrospectives are very important. Some think the most important thing.

    I am not sure what you mean by "try". I am guessing that you mean the exercise where we talk about 'stop', 'continue' and 'try'. Try meaning new things that we will try as a team.

    In a broad sense, I think you mean removing one good-sized impediment each sprint. And in that case, I agree strongly.

    Thanks,
    Joe

    ↓

Leave a Reply