In another brilliant article for the New Yorker, Atul Gawande writes about how Peter Pronovost, a doctor at John Hopkins Hospital, has been proving the value of the humble checklist in situations which are “now too complex for clinicians to carry them out reliably from memory alone.”
Eliminating bugs (both kinds)
The example Pronovost started with was line-infections (getting infected in hospital from the tubes they stick into you). He introduced a simple checklist for the steps that should be followed to avoid infecting patients when putting in a line, and gave the nurses the necessary backing to enforce compliance.
Pronovost and his colleagues monitored what happened for a year afterward. The results were so dramatic that they weren’t sure whether to believe them: the ten-day line-infection rate went from eleven per cent to zero.
Being a geek by trade (and given to making dubious connections) this made me think of the better XP projects I’ve been on where our bug rate has been, effectively, zero. Is there a coincidence here?
Thinking about it, most of the tests I write have a checklist feeling about them. Does this code reject trades that are over the risk limit? Check. Does this code show that the flight has been reserved? Check. We also have checklists around the code, most of which are automated these days. Does this application compile cleanly? Check. Does this application deploy and run properly? Check. Have we kept all our old functionality working? Check. Does our code pass all our style guidelines? Check. And good teams have checklists around the project. Has this piece of work been done? Check. Is this the most important thing to work on today? Check. And so on. (OK, so the point is getting weaker here, but it’s not completely false).
The checklists provided two main benefits, Pronovost observed. First, they helped with memory recall, especially with mundane matters that are easily overlooked in patients undergoing more drastic events. […] A second effect was to make explicit the minimum, expected steps in complex processes. Pronovost was surprised to discover how often even experienced personnel failed to grasp the importance of certain precautions. […] Checklists established a higher standard of baseline performance.
As I write this, I’m starting to think that the checklist is a more appropriate metaphor than test, specification, or behaviour. I find that the act of just listing what a unit of code is supposed to do drives out all sorts of interesting functionality questions and edge cases, and that writing checklist-style tests gives me a place to do this, a structure that helps me focus my understanding of the domain and my experience as a developer.
So why doesn’t everyone do it?
The article then goes on to talk about the slow take-up of checklists in US hospitals.
We have the means to make some of the most complex and dangerous work we do—in surgery, emergency care, and I.C.U. medicine—more effective than we ever thought possible. But the prospect pushes against the traditional culture of medicine, with its central belief that in situations of high risk and complexity what you want is a kind of expert audacity—the right stuff, again. Checklists and standard operating procedures feel like exactly the opposite, and that’s what rankles many people.
This sounds like some of the objections that I’ve heard against TDD; that it’s Consultant Snake-Oil, and that it inhibits creativity (this from a tester!).
It’s ludicrous, though, to suppose that checklists are going to do away with the need for courage, wits, and improvisation. The body is too intricate and individual for that: good medicine will not be able to dispense with expert audacity. Yet it should also be ready to accept the virtues of regimentation.
While as coders we will never face the drama of a medical emergency, our fundamental problem is complexity and most everyone works on systems where we simply cannot remember everything. A colleague says that he took up TDD when he went into business on his own and simply couldn’t afford to let stupid mistakes block making progress (whilst building complex and innovative applications). That requires the right kind of humility; when we disagree with the machine we lose.
There’s lots more good stuff in the article about some of the techniques Pronovost has used to get hospitals to adopt checklists: finding unthreatening ways to make his point, getting just enough funding to make an experiment worth the trouble, empowering the right people (nurses), and getting commitment on the ground from executives. The results of a pilot study in Michigan were spectacular.
In the Keystone Initiative’s first eighteen months, the hospitals saved an estimated hundred and seventy-five million dollars in costs and more than fifteen hundred lives. The successes have been sustained for almost four years—all because of a stupid little checklist.
Hospitals have spent millions on high-tech solutions, many of which have not been as effective as a sheet of paper. It makes me think of the distance between some of the expensive software tools I’ve seen (and seen mandated to be used) and the ubiquitous (in XP at least) index card.
An Institute of Software Delivery
Here’s Provonost again.
“The fundamental problem with the quality of American medicine is that we’ve failed to view delivery of health care as a science. The tasks of medical science fall into three buckets. One is understanding disease biology. One is finding effective therapies. And one is insuring those therapies are delivered effectively. That third bucket has been almost totally ignored by research funders, government, and academia. It’s viewed as the art of medicine. That’s a mistake, a huge mistake. And from a taxpayer’s perspective it’s outrageous.” We have a thirty-billion-dollar-a-year National Institutes of Health, he pointed out, which has been a remarkable powerhouse of discovery. But we have no billion-dollar National Institute of Health Care Delivery studying how best to incorporate those discoveries into daily practice.
I’m not sure how this issue maps to the software industry. There’s a history of attempting to fix the larger issues of software delivery, much of it sponsored by large organisations. It’s possible that this kind of work, or its misapplication, has raised the barriers to the adoption of solutions like checklists because of the legacy of process styles that try to squeeze the human element out of development. There’s also academic research on all sorts of topics, such as systems, formal methods, and modelling. What I see less of (and this may well be raw ignorance), is research that helps me do better in my day-to-day work. I think one of the most important contributions of XP was that it was a development methodology that actually considered the act of writing code.
Dave Nicolette has a nice post called Good-enough today beats complete next year about
how Agile helps to focus development on delivering something valuable in time for the business, instead of waiting for the full, perfect solution.
I think it’s important also to distinguish different kinds of quality. The clue is in the phrase “well-graded dirt road”, which means that it doesn’t do much but it is well built within its parameters. This is not the same thing as laying asphalt on mud, which might be as quick to build and looks good, but will be a source of misery and expense as long as it exists—and will be in the way when it’s time to build something more substantial.
“The Edge” online magazine posts a question every year to lots of interesting people. The strapline this year is:
Science is based on evidence. What happens when the data change? How have scientific findings or arguments changed your mind?”
A major time-sink.