Why understanding Deming PDSA will help you solidify the foundation of Agile

How do you argue with someone who would pretend that iterative development process is not as solid as Waterfall process (you can read my past article “Waterfall: a method that never was” when it will be restored from backup) ? In big organizations, not so few old-school project managers are still convinced that Agile is just bullshit marketing that will fade with time. Unfortunately, they risk to be apparently right as many projects supposedly agile are far from being better than Waterfall and end miserably. That’s why some people have been switching from one methodology to another over the years hoping for the better (RUP to Scrum and last trend is to Kanban … what’s next ?) but it is probable that they will also fail because the result will be more or less the same.

There may be many reasons given about the failure of these projects not really been agile, but as for me the number one reason is that people do not understand the why of iterative process since they have never been explained where it originates from : Deming Wheel or Deming PDSA. That’s what I’ll try to do here in depth.

Though PDCA (Plan – Do – Check – Action) is the most widespread acronym it was a bad recast of the original Deming’s wheel because Deming didn’t like the term “check” and later pushed for his alternative PDSA (Plan – Do – Study – Act). Deming’s aim was to adapt Shewhart’s 3 steps cycle (meant for mass production manufacturing) :

- Specification
- Production
- Inspection

into a more business-wide process with PDSA :
- Plan
- Do
- Study
- Act

Whether 3 or 4 steps, what really matters here is that these cycles should actually be viewed as a scientific process to acquire knowledge. The proof is in Shewhart’s seminal book “Statistical Method from the Viewpoint of Quality Control” (with foreword by Edwards Deming) p.44:

“It may be useful to think of the three steps … as steps in the scientific method. In this sense, specification, production and inspection correspond respectively to making a hypothesis, carrying out an experiment, and testing the hypothesis. The three steps constitute a dynamic scientific process of acquiring knowledge.”

What’s also as important is that in the real world, as Shewhart puts it there is no “exact science” but “only probable science”. He then wrote:

“The third step can not be taken by simply inspecting the quality of the objects as objects, but instead must be taken by inspecting the objects in a sequence ordered in relation to the production process.”

Therefore:

“These three steps must go in a circle instead of a straight line as shown schematically in fig.10″:


You can now tell all the skeptics about Agile that for sure Iterative process is based on sound Scientific Principle and if they ever doubt that Agile doesn’t relate tightly to PDCA here’s a nice illustration:

But as an other consequence, Iterative process aim should be more about delivering less craps (quality software) than about maximizing throughput of (crappy) features. That’s where the deception for Managers can arise as they were sold by most Agile Marketing shops that Iterative Process is about delivering faster. So as to fulfill this expectation, you can bet developers will be more focused on this outcome than on the process of delivering quality software. As Deming said:

“We should work on our process, not the outcome of our processes.”

That’s where the difference between PDCA and PDSA fundamentally resides: Deming’s PDSA is not just about changing one letter in an acronym, Deming’s PDSA is about a difference in perspective. Deming’s PDSA targets process improvement, not the outcome of a process. 

For example, using WIP limits in Kanban to show the client that they shouldn’t ask for more is just about using it as defensive tool not really about process improvement except at first stage for stabilizing the process or in Deming’s parlance putting the system under control (making variance less chaotic). But once it’s done, process improvement should be engaged : what about experimenting different WIP limits (to decrease the variance)? But even decreasing variance is a poor objective so what about finding the root causes of the discrepancies between Business expectation and Software Development ? More often than not, the software architecture is obsolete – if there was any. This is what Continuous Improvement for acquiring Knowledge of the system is about : it’s about process which means design in software.

In conclusion, do not forget that whatever methodology, agile people alone won’t bring any real process improvement if not coopting with people mastering software engineering (a boring discipline that tends to be forgotten in the agile enthusiastic environment) to ask the right questions and work on the root causes. This in the end will allow to not just make good (without bugs) but hopefully great softwares (that delight consumers) faster because the Architecture will be able to sustain it – otherwise you risk to do design by coincidence which may be unmaintainable. Delivering just faster or more often crappy softwares won’t make clients happier.