Stephen Freeman Rotating Header Image

June 15th, 2008:

Test-Driven Development. A Cognitive Justification?

It’s been a busy week. Michael Feathers has an interesting post on the nature of Test-Driven Development, to which Keith has responded. I think Michael overstated my position on “most” people (it was probably a bar discussion) but over the years I’ve seen a lot of TDD code that doesn’t look right. Incidentally, Tim Mackinnon, who was there, tells the origin of Mocks story at the bottom of this page.

With that out of the way, I’d like to get to the real point of this posting

A Cognitive Justification for Test-Driven Development

Two influences coincided for me at XP2008 this week: Dave Snowden talking about social complexity, including current understanding of the how the mind works, and Naresh Jain pairing to understand different people’s approaches to Test-Driven Development.

Dave has spent a lot of time exploring how decision-making happens. In particular, it turns out that people don’t actually spend their time carefully working out the trade-offs and then picking the best option. Instead, we employ a “first-fit” approach: work through an ordered list of learned responses and pick the first one that looks good enough. All of this happens subconsciously, then our slower rational brain catches up and justifies the existing decision—we can’t even tell it’s happening. Being an expert means that we’ve built up more patterns to match so that we can respond more quickly and to more complicated situations than a novice, which is obviously a good thing in most situations. It can also be a bad thing because the nature of our perception means that experts literally cannot receive certain kinds of information that falls outside their training, not because they’re inadequate people but because that’s how the brain works.

Part of Dave’s practice is concerned with breaking through what he calls this “Expert Entrainment”. He has developed exercises to shuffle our list of response patterns and allow other ideas to break through the crust of skills we’ve worked so hard to acquire. One motivation for doing this is to stop experts jumping to a known solution when they haven’t really understood the situation.

Naresh, meanwhile, is on a mission to pair program with the world to understand how different people approach Test-Driven Development, with an example problem that he uses with everyone. My preference these days is to start with a very specific example of the use of the system and then, as I add more examples, extract structure by refactoring. As we talked this through, Naresh described another programmer who noticed that the problem was an instance of a more general type of system and coded that up directly, there was nothing in his solution that included the language of the example. The other programmer had used his expertise to recognise an underlying solution and short-circuit the discovery process—that’s why we claim higher rates for experience. This programmer was right about his solution, so why did the leap to a design bother me (apart from my own Expert Entrainment)?

Then it struck me, Test-Driven Development, at least as practised by the school that I follow, progresses by focussing on the immediate, on addressing narrow, concrete examples. Don’t worry about all those ideas buzzing around your head for how the larger structure should be, just make a note and park them. For now, just do something to address this little concrete example. Later on, when you’ve gathered some empirical evidence, you can see if you were right and move the code in that direction.

I think what this means is that Test-Driven Development works (or should do) by breaking our first-fit pattern matching. It stops us being expert and steam-rolling over the problem with, literally, the first thing that came into our minds. It forces us out of our comfort zone long enough to consider the real requirements we should be addressing. Even better, starting with a test forces us to think first about the need (what’s the test for that?), and then about a solution that our expert mind is so keen to provide.

Just in case you missed that (and it took me a while to see it), it makes a cognitive difference whether you write the tests first or the code.

The best supporting evidence is Arlo Belshee’s group that implemented Promiscuous Pairing. They found empirically that they were most productive when switching pairs every couple of hours, contrary to what anyone would expect; their view was that were taking advantage of constantly being in a state of “Beginner’s Mind”. Of course, to make TDD work in practice, we still need all that expertise underneath to draw on but to support, not to control.

Personally, I’m constantly surprised at the interesting solutions that come up from being very focussed on the immediate and concrete, with a background awareness of the larger picture. By letting go, I discover more possibilities. Very Zen.