Dr. Royce and Waterfall

In 1970, Dr. Winston Royce wrote the paper that defined Waterfall.  It was probably beig done by others at the time, but the paper is considered definitive.

Here is the paper: http://agileconsortium.pbworks.com/w/page/52184647/Royce%20Defining%20Waterfall

He identified 5 things that must be added to reduce most of the risks of doing waterfall.  And I will comment on how Scrum addresses these.

1. Program Design Comes First 

I think he may have been mostly right.  But I would rather say that ‘solution design’ comes first. Meaning that we must truly try to understand the customer’s real problem (not what he says it is) and try to design a full solution to that problem.  Not just, for example, software, but the full solution. Assuming the product is (mainly) software, we  better understand where our product plays in the fuller solution.

We should say quickly, as Lean Start-up would say, that we should assume that we have gotten it wrong, and quickly start to seek feedback, ideally via an early first release.

Scrum is only a bare framework.  Scrum does not address this issue directly.  Implicitly Scrum is saying, “it is hard to know whether your product will be good, so, build incrementally and try to enhance your feedback, to prove to yourself that the product will support the solution.”

Let me add this: Some people who are doing ‘cowboy agile’ in fact do not do enough up-front thinking, in my experience.  Yes, it is fair to say that we learn more from working software than by just ‘thinking.’  But some don’t ‘look before they leap.’

This is a hard thing to discuss in the abstract, because how much to think up-front will vary, depending on many details in your specific situation (for example, the experience level of the implementers).

I have also seen Scrum teams get into analysis paralysis, where they are thinking too much up-front.  In my experience, it is well worth the team’s time to discuss continually, “well, how much should we think up-front about this piece?”  They will get better at making intuitive judgments about this, that are appropriate, for their specific work.

2. Document the Design

Agile and Scrum do not have Dr. Royce’s level of confidence in documentation.

And clearly some ‘agile’ people seem to believe in no documentation. (I strongly disagree.)

Agile and Scrum see, or assume a greater urgency to meet ‘time to market’ demands. Typically a high focus on documentation leads to long analysis paralysis periods.

Agile and Scrum do agree that knowledge must be shared and saved. But rather than put so much energy sharing knowledge solely through documentation, the community now suggests doing it many other ways. Here are two: (1) incrementally build working product, and (2) lightly documented automated tests (where at all possible).  Agile and Scrum people tend to prefer cards (middle-level summaries), white boards, large sheets with drawings in team rooms, etc.

Note again: These are in addition to some level of shared documentation (in the more traditional sense).  This documentation includes diagrams especially, and lists, and tables.  Things that people will read and discuss.

So, information is shared more, and the sharing is actually better.

Speaking for myself, I do agree with Dr Royce’s statement that many implementers tend not to write even basic documentation.  And I agree with his statement that much of the purpose of documentation is for use by people later.  So, those needs must be carefully considered and addressed.

This results, if done professionally, in two things:
1. We in agile write less documentation than at least I used to do, when doing ‘waterfall.’
2. We ask many people in the Team to write more documentation (in a wiki or somewhere) than they want to write.  This means we have to explain frequently WHY they should want to write this documentation.

3. Do It Twice

Yes, Dr. Royce said “build it once, throw it away, and then build the real thing”.  His son later complained that no one really did this, so no one was really doing waterfall the right way…

The basic problem is that we are always learning, even in the last stages of the work. And that learning needs to be fed back into the final product. This was his recommendation for doing that in waterfall.  And few people do it.  No wonder the waterfall products are seldom what the customers really need.

Scrum solves this differently, by getting frequent feedback, and continually refactoring the current product to align with the latest knowledge.  So, having ‘working product’ every sprint is key.

This means, yes, Virginia, there is ‘re-work.’  But probably not any more re-work than Dr. Royce proposed.  In fact, almost surely less.  And the Team should be asking: ‘How could we have avoided some of this re-work?’  Sometimes they will come upon ideas to get better next time (next sprint).

We are not suggesting or expecting to arrive at a state of ‘perfect knowledge up-front’.  This will never happen. Still, we can learn to consider some things before we leap. Example: Is our design for this story consistent with the rest of the design and also appropriate versus our complete and current understanding of what the overall solution needs to be?

4. Plan, Control, and Monitor Testing

Dr. Royce has some good ideas about testing.  Some of them have been superseded and some new things have come onto the scene as well.

The key thing to say, though, is that Scrum agrees with the importance of testing. Scrum says: let’s do it from the very beginning (well, at least from the first Sprint).  And agile adds: let’s start automating the testing iteratively and incrementally.  In part, to tighten the feedback loop and to reduce the cost of implementing the changes we get from the feedback.

5. Involve the Customer

Yes!  Very important.  Again, Scrum considers this such an important idea that we do it in the form of the Product Owner, who is an essential daily part of Scrum.  In my opinion, the PO role idea should get ‘the customer’ involved in a much more practical and useful way that the ideas that Dr. Royce proposes for waterfall.


Much more to say, but let me keep this post short.

Early in the paper Dr. Royce says: “I believe in this concept [waterfall], but the implementation described above is risky and invites failure.”  I don’t know, but I think he meant “Compared to cowboy coding [no process], waterfall, with the 5 ‘additions’ that I will propose, seems a lot better.”  And given many factors at the time, his idea, at least as I compared it, may have been correct.

Now let me quote Dr. Royce’s last lines about waterfall:

“[This] summarizes the five steps that I feel necessary to transform a risky development process into one that will provide the desired product. I would emphasize that each item costs some additional sum of money. If the relatively simpler process without the five complexities described here would work successfully, then of course the additional money is not well spent. In my experience,the simpler method has never worked on large software development efforts and the costs to recover far exceeded those required to finance the five-step process listed.” [emphasis added]



« « Scrum vs Waterfall : How do they Compare? || Why Do Scrum? » »

2 thoughts on “Dr. Royce and Waterfall

  1. Joseph Osako

    The biggest problem with Waterfall isn’t even with what Royce wrote, but with what most people thought he wrote. It seems that most of the readers of the article never looked past figure 2, which is ‘cascade’ diagram which every textbook and paper on design since has copied. The first thing that the paper says after presenting the ‘classic’ Waterfall diagram is “this won’t work, ever” (verbatim: “I believe in this concept, but the implementation described above is risky and invites failure”). He goes on to say that while every software process goes through most of these steps in something like that order, trying to force the project into a linear, one-way route typical of physical engineering practices was an impossible task.

    In other words, Royce wasn’t establishing what later came to be known as Waterfall; he was trying to demolish it before it could spread, but unfortunately ended up spreading it further.

    The rest of the paper is actually much closer to Agile than most people credit it for. In particular, he emphasizes the need for ‘spiraling’, of being able to go back an arbitrary number of steps when an unforeseen error or change arises. While his emphasis on documentation and Big Design Up Front are at odds with current thinking, you have to consider the time he was writing – there just wasn’t enough experience with software design for him to see the flaws in those approaches. Also. the projects that were being developed then (primarily banking, accounting, and payroll systems, and military or aerospace control systems) were mostly of sorts which were a lot more amendable to BDUF, and required vast amounts of documentation regardless of anything else because of regulatory compliance issues – Agile, for all its advantages, simply isn’t applicable for those sorts of projects, because of the need for a stable, well-documented and largely immutable codebase.

    Finally, while his insistence on prototyping and discarding the temporary version is dated, careful reading of the purpose he gives this gives me the impression that he was blindly grasping towards something akin to modern ideas such as continuous integration, fast turnaround, and so forth. He just didn’t have a context to fit these ideas into beyond the existing physical engineering ideas of prototypes and pilot designs, and there was no way he could have crossed the chasm between those two sets of ideas.

    1. Joe Little Post author

      Hi Joseph,
      I like your comments. And mostly agree. Which is the more important thing.
      A few comments:
      – I think at the time, this was what Dr. Royce thought was the best way. He was not happy with it, and did not recommend it strongly, but I think he wrote the paper to ‘make it better’ rather than to destroy it.
      – Other people have said that Agile did not become ‘possible’ until we had certain programming ideas and a concept of Object Oriented Design. And some tools (eg, CI). Perhaps this is so; or perhaps it became more obvious that waterfall was failing and ‘there has to be a better way!’. Which is when most things are invented. And machines needed to become cheaper (many were working with, at the time, very very expensive mainframe machine…imp issue).
      – 20-20 hindsight, we all should have ‘invented’ agile much sooner. But of course, that is true for every good invention.
      – His advice is: Do It Twice. I think he is serious. That is, do not think of the initial version as ‘a prototype’ or ‘a temporary version’, but think of it as the real thing. Still, invariably, it will not be good enough. Perhaps you ‘release’ it to some degree, to get feedback. But you have already planned to Build It Again. And you do. This is very like, IMO, the Lean Start-up idea of a major pivot after the first release.

      Many thanks for your remarks.
      Regards, Joe

Leave a Reply