Stephen Freeman Rotating Header Image

June, 2008:

Test-Driven Development. A Cognitive Justification?

It’s been a busy week. Michael Feathers has an interesting post on the nature of Test-Driven Development, to which Keith has responded. I think Michael overstated my position on “most” people (it was probably a bar discussion) but over the years I’ve seen a lot of TDD code that doesn’t look right. Incidentally, Tim Mackinnon, who was there, tells the origin of Mocks story at the bottom of this page.

With that out of the way, I’d like to get to the real point of this posting

A Cognitive Justification for Test-Driven Development

Two influences coincided for me at XP2008 this week: Dave Snowden talking about social complexity, including current understanding of the how the mind works, and Naresh Jain pairing to understand different people’s approaches to Test-Driven Development.

Dave has spent a lot of time exploring how decision-making happens. In particular, it turns out that people don’t actually spend their time carefully working out the trade-offs and then picking the best option. Instead, we employ a “first-fit” approach: work through an ordered list of learned responses and pick the first one that looks good enough. All of this happens subconsciously, then our slower rational brain catches up and justifies the existing decision—we can’t even tell it’s happening. Being an expert means that we’ve built up more patterns to match so that we can respond more quickly and to more complicated situations than a novice, which is obviously a good thing in most situations. It can also be a bad thing because the nature of our perception means that experts literally cannot receive certain kinds of information that falls outside their training, not because they’re inadequate people but because that’s how the brain works.

Part of Dave’s practice is concerned with breaking through what he calls this “Expert Entrainment”. He has developed exercises to shuffle our list of response patterns and allow other ideas to break through the crust of skills we’ve worked so hard to acquire. One motivation for doing this is to stop experts jumping to a known solution when they haven’t really understood the situation.

Naresh, meanwhile, is on a mission to pair program with the world to understand how different people approach Test-Driven Development, with an example problem that he uses with everyone. My preference these days is to start with a very specific example of the use of the system and then, as I add more examples, extract structure by refactoring. As we talked this through, Naresh described another programmer who noticed that the problem was an instance of a more general type of system and coded that up directly, there was nothing in his solution that included the language of the example. The other programmer had used his expertise to recognise an underlying solution and short-circuit the discovery process—that’s why we claim higher rates for experience. This programmer was right about his solution, so why did the leap to a design bother me (apart from my own Expert Entrainment)?

Then it struck me, Test-Driven Development, at least as practised by the school that I follow, progresses by focussing on the immediate, on addressing narrow, concrete examples. Don’t worry about all those ideas buzzing around your head for how the larger structure should be, just make a note and park them. For now, just do something to address this little concrete example. Later on, when you’ve gathered some empirical evidence, you can see if you were right and move the code in that direction.

I think what this means is that Test-Driven Development works (or should do) by breaking our first-fit pattern matching. It stops us being expert and steam-rolling over the problem with, literally, the first thing that came into our minds. It forces us out of our comfort zone long enough to consider the real requirements we should be addressing. Even better, starting with a test forces us to think first about the need (what’s the test for that?), and then about a solution that our expert mind is so keen to provide.

Just in case you missed that (and it took me a while to see it), it makes a cognitive difference whether you write the tests first or the code.

The best supporting evidence is Arlo Belshee’s group that implemented Promiscuous Pairing. They found empirically that they were most productive when switching pairs every couple of hours, contrary to what anyone would expect; their view was that were taking advantage of constantly being in a state of “Beginner’s Mind”. Of course, to make TDD work in practice, we still need all that expertise underneath to draw on but to support, not to control.

Personally, I’m constantly surprised at the interesting solutions that come up from being very focussed on the immediate and concrete, with a background awareness of the larger picture. By letting go, I discover more possibilities. Very Zen.

Travel to XP2008?

In preparation for my (first) tutorial at XP2008, I’ll be arriving at Shannon airport at about 7:30, Tuesday night. If anyone wants to share a ride to Limerick, please let me know. Comment on this blog if you don’t have my address.

Comedy is close to tragedy

The BBC is broadcasting Harrison Birtwistle’s new opera “The Minotaur” this weekend. Personally, I’m very pleased they’re doing this since it’s a tremendous work and the production was first rate. On the other hand, look at the schedule.

7:35 pm The Minotaur Part One: World premiere of Harrison Birtwistle’s new opera from the Royal Opera House, Covent Garden. A ship bound for the island of Crete carries the innocents to their slaughter.
9:00 pm Have I Got a Bit More News for You A bit more of the satirical news quiz with Ian Hislop and Paul Merton, panellists Ian McMillan and Kate Silverton, and guest host Jeremy Clarkson.
9:40 pm The Minotaur Part Two: World premiere of Harrison Birtwistle’s new opera from the Royal Opera House, Covent Garden. Theseus confirms to Ariadne that he is determined to face the Minotaur.

The end of Part One is one of the most dramatic (and scary) scenes I’ve seen on stage in a while. So, either this is a brilliant piece of scheduling intended to play with tension and relief in drama or, um, it isn’t.

Grumbling aside, I’m just glad that there’s still some good public service broadcasting in this country.

Do you know where your iPhone is?

Scary security story about personal data not being flushed on iPhones at

A few days ago, I posted a discovery in that personal data remains intact (in deleted portions of the file system) following a full iPhone restore. As it turns out, Apple themselves may not have been aware of this. Thank goodness, otherwise identity theft might actually be, like, hard. A detective from the Oregon State Police, whom I’ve verified, notified me this afterrnoon that an out-of-the-box refurbished iPhone he purchased directly from Apple contained recoverable personal data. This included email, personal photos, and even financial information that he was able to recover using my forensic toolkit. Needless to say, the original owner was quite surprised. He informed me that the device had been returned to Apple under a warranty exchange only a few months ago, suggesting that Apple has been using an insecure refurbishing process for the past year. Here are some blurred screenshots of just some of the data recovered:

"Officers 'to use own judgement'"

Can we hope this is the first sign of rolling back Whitehall’s government by targets?

Four police forces are to abandon government targets and allow officers to decide whether to make arrests.

From BBC news