Stephen Freeman Rotating Header Image


Husbanding willpower

I picked up the New Scientist for my in-flight reading on the way to JavaZone and came across a fascinating article.

For those who don’t have a subscription, it turns out that willpower is a limited resource, like exercising a muscle. Different people have different limits, but when they’ve used it up they need time to recover. Worse, the relevant part of the brain may also be worn out by making difficult decisions and coping with stress. This is why it’s a bad idea to try to reform all your habits at the same time, there isn’t enough mental strength to go round.

The implications for us Agilistas are interesting. It explains the value of ceremony and automation that we emphasise so much, they relieve us from wasting willpower on known practices. It also suggests a validation for pair programming in that it allows one member of the pair to recharge their motivational batteries whilst the other pushes on. Another interesting result, from Peter Gollwitzer at NYU, is that doing task breakdown for an activity helps to get it done,

Planning can turn a difficult conscious decision into an unconscious habit, which makes the whole process faster and more efficient without depleting energy levels.

which sound like a good reason to crack open another pack of index cards.

Two more relevant points. First, willpower takes physical energy. Spending long hours slumped in front of the keyboard means that much of that work will be done when the brain is too tired to think things through. Second, willpower can be strengthened by practice. Exercising self-control in one area can boost it in others, which suggests to me that the benefits of building a high-quality culture will be better than linear.

The research also reinforces Roy Osherove’s point on the difficulties of getting TDD adopted, even if we differ on the solution.

Perhaps this means that there could be a measure for the hidden costs of organisational drag, what Ivan Moore and Rachel Davies referred to as Gumption Traps. If just making it to your cubicle burns up a significant fraction of your store of willpower, then there’s less left for writing great systems; that’s why I think the regime at DEC SRC was so remarkable.

Perhaps it might also explain why so many geeks who write perfectly clean code live in such a mess.

P.S. The New Scientist web site currently includes this infinitely depressing story about how, in a world that can find nearly a $1G to save the banks, we can’t save the cod.

Fishing vessels on the Grand Banks of Newfoundland are this week destroying the best hope for years that the region’s cod fishery, once the world’s most abundant, might yet recover.

And at a meeting in Vigo, Spain, governments have rejected a simple measure that might have given the cod a fighting chance.

That’s why I can’t read it all the time.

Great assistants help everyone

Johanna Rothman has a nice post about why people with responsibilities need assistants. As she points out, most organisations have been stripping out administrative support except for the most exalted positions, which makes sense until one looks at the bigger picture. Is it really better that people who are supposed to be busy with activities that generate income for the company spend their time filling in orders for stationery?

One of my (multiple) rants is about my brief experience at Digital Corp’s late Systems Research Center. Their technical recruiting was pretty tough but so, I guess, was their recruitment for all the other staff. They had the best administration staff I’ve ever seen. They would perpetrate unprovoked acts of forethought and helpfulness when you weren’t looking, same with the system administrators.

This works on all sorts of levels. First, if you’ve just spent a great deal of effort recruiting top-class researchers, it doesn’t make sense to have them spend time battling the corporate bureaucracy. Second, and deeper, the ethos in the building was that stuff would just work, so the technical staff would not be diverted from their real jobs—and, by implication, there would be no excuses for not doing good work. In contrast, at another international research lab, it took me months to get my expenses paid because the relevant administrator could not figure out which way exchange rates worked (more Dollars than Pounds at the time).

Like many such systems, it’s hard to understand or believe how much of a difference getting things right makes until you’ve experienced it; I expect this is how it feels at a first-rate Toyota plant. The rest of us have to learn to cope with the Gumption Traps waiting for us in the “real” world. In the distance, we hear the Siren call of the Pragmatic Fix.

Microcosmographia Academica

For reasons too complicated to explain, I’ve ended up with a copy of University Politics which includes a full reprint of F. M. Cornford’s satire on organisations Microcosmographia Academica, continuously in print since 1908. Of course, now the 15 page pamphlet comes with a 100 page scholarly introduction.

Anyway, it’s all still true and those of us who are “young men in a hurry” 1 trying to change things need to keep in mind the facts of organisational life. Here’s a sample:

Since the stone-axe fell into disuse at the close of the neolithic age, two other arguments of universal application have been added to the rhetorical armoury by the ingenuity of mankind. They are closely akin; and, like the stone-axe, they are addressed to the Political Motive. They are called the Wedge and the Dangerous Precedent. Though they are very familiar, the principles, or rules of inaction, involved in them are seldom stated in full. They are as follows.

The Principle of the Wedge is that you should not act justly now for fear of raising expectations that you may act still more justly in the future — expectations which you are afraid you will not have the courage to satisfy. A little reflection will make it evident that the Wedge argument implies the admission that the persons who use it cannot prove that the action is not just. If they could, that would be the sole and sufficient reason for not doing it, and this argument would be superfluous.

The Principle of the Dangerous Precedent is that you should not now do an admittedly right action for fear you, or your equally timid successors, should not have the courage to do right in some future case, which, ex hypothesi, is essentially different, but superficially resembles the present one. Every public action which is not customary, either is wrong, or, if it is right, is a dangerous precedent. It follows that nothing should ever be done for the first time.

Conford was one of the generations of academics who somehow managed to convert Cambridge from a combined training school for Anglican clerics and finishing school for the Gentry into something like a modern Victorian university. It might be that the university’s Byzantine organisation is what gives it the flexibility to survive. Most of rest of us, however, don’t belong to organisations that can wait 100 years to catch up with the opposition, so we’d better get a move on.

1) “Young men in a hurry” come in all ages and genders.

What is going on out there? A Narrative Inquiry project

Joseph Pelrine and I are involved in (or is that “committed to”?) starting up a Narrative Inquiry project with the Agile Alliance under their Agile Narratives programme. This one will be done jointly with Cognitive Edge using the Cynefin approach, here’s their announcement.

I’ve started a Yahoo Group for people who are interested.

What is going on out there?

A proposal for a Narrative Inquiry project within the Agile Narratives Program


How does the Agile Alliance find out what’s important to its members, and highlight interesting new ideas amongst all the noise? The activity of finding out is called Sense Making. This proposal is for an experiment to understand how well one approach to Sense Making works and can be applied to the Agile community.

The Agile world is changing as adoption by individuals and organisations grows rapidly. It used to be possible for individuals more or less to know what was going on, but the community has long been too large for that kind of personal network. How do Agile practitioners, novice and experienced, make sense of current practice in the discipline?

One technique is to use personal stories from the community, managed as semi-structured data, to support quantative analysis and to help understand individuals’ concrete experiences. The Agile Narratives Program has been investigating one approach to this technique. This proposal is to start a complementary project in collaboration with Cognitive Edge using their Cynefin methods and SenseMaker tools.


We will set up a programme to gather and index brief stories from members of the community about their experiences of adopting and working with Agile software development practices.

Bootstrap. We will start with a session, run at the London eXtreme Tuesday Club, to gather “indexing” topics, questions that the storytellers themselves will use to categorise the stories that they contribute. Cognitive Edge will then set up an initial data-gathering site based on those questions. We can prompt for contributions within the XtC and various Agile groups on the internet to seed the contents of the database. We might also be able to use some of the content from the existing Agile Narratives database.

Pilot. We will run a pilot event at XP2008, based on this data, to trial techniques for gathering and analysing further stories, in the context of a public event. Dave Snowden (founder of Cognitive Edge) will be a keynote at that conference, which will help to attract participation. This pilot will test our approach and should provide enough initial data to experiment with quantative analysis.

First run. We will run activities at with Agile2008 to raise awareness of the program and to gather stories from the much larger group, based on the experiences from the Pilot. We might have enough data by then to attract attention with some early findings, or be able to show how the results change over the week.

Ongoing. Once the project is established, we intend to keep it active as a place to contribute and find stories about Agile practice “in the field”, about practitioners’ real experiences. From time to time, we can re-analyse the data to report trends and surprises.

Suddenly I've become less sympathetic to non-technical managment

I bumped into an old colleague recently who told me her tale of woe.

She’d been consulting for a while with a small business that provides quite a successful online service. The company had been bought out and, for a while, she improved their naive development practices while coping with complexities such as a new CTO who lived abroad and regularly dropped out of contact. In the end, she moved on to a new contract.

Sometime later, the CEO got in touch. He’d backed off some of his worst ideas, dropped some of the management and, eventually, hired her back as CTO. Good news.

On her first day, the lead developer threatened to quit unless he was allowed to start a proposed rewrite that day, another developer joined the mutiny. The CEO phoned after lunch and, with deepest regret, her tenure was over. The thing to remember about this situation is that the development team were mainly hired fresh out of college and only know this one company, whereas my colleague has lots of experience and is a name in the field. In her view, the codebase is bad, but not beyond rescue. My guess is that the company will burn a lot of cash on version 2 while its customers get fed up waiting for new features or patches to a neglected version 1. It’s a very high risk strategy for a small company.

How could the CEO fall for this? Presumably he knows a lot about the business the company services, but he has no background in the mechanics of what makes his product work. He has no instinct for when the developers, with the best of intentions, are getting carried away. So he’s taken what looks like the lower risk strategy, keep the people who got them this far, when in practice it’s the opposite. He’s also shown that he’s hostage to his developers which, much as I want to stand up for the underdog, is not a safe policy for either side. If he has any survival instincts at all, he’ll be looking for alternatives. Version 2 had better be a winner.

Update: Thinking further, I think it might even be worse, since the CEO didn’t see this coming, he hired a CTO without knowing how various core people would react and he didn’t have enough of their respect to make them listen. This is in a O(30) person company, not a mega-corp.

Potemkin Agile

I’ve been meaning to write this post for a little while and now I feel triggered by Keith Braithwaite’s especially grumpy contribution.

I’ve had a few discussions around the “Has Agile Lost Its Mojo” session, including with Keith. One person called me an elitist, which is an interesting term of disapproval around an investment bank. Clarke Ching wrote that he bailed out of the related session after five minutes because he was so upset.

First the “Mojo” session, which was intended to be a contentious topic with a cute title that would get people talking on their way to the pub. In that respect it seems to have succeeded. It was also intended to flag what looks like a shift in the community as ideas that were treated as ludicrous less than ten years ago have become accepted, and even dogma, in some communities. Is this the moment where the venture capitalists oust the founders to bring in experienced management? Maybe it is judging by the number of organisations with proper sales people who have started using Agile terminology.

It’s true, as Keith points out, that the Agile manifesto is largely platitudes—except that there are still plenty of organisations that don’t act as if it were true, particularly the one about people over process. But I claim that some of the success that Keith, Clarke, and others have been having (apart from the fact that they know what they’re doing) is because the hot-house clique (including Keith) and others went out and made it work, and generated enough noise to make terms like “incremental” acceptable in polite conversation. Very little of the kind of improvements we’re introducing today could not have been introduced before, so something must have changed. I, for one, would be sorry to see the extremists of the Agile movement wither away because, if nothing else, that’s where the ideas get to be “well-tried”. As Dave Snowden just wrote in a much heavier post than this,

Multiple small initiatives showing that there is a different way of doing things are vital, and people prepared to make sacrifices of convention to establish them are to be praised. The witness of community is a part of the history of humanity and one that continues and needs to continue today.

He also has some warnings about Model Communities, but that’s for another day.

As for the “Compromised Agility” session. Again, this was intended to be contentious. I know the presenters and they appear to have been attracting favourable attention from some very senior people at their current client because they’re offering real value instead of faffing around like their competitors. To quote from the client’s CFO, “This is the first time I’ve visited a team where everyone clearly knows exactly what’s going on in the project.” They got the job because they don’t compromise on the stuff they think is important and they managed to find a client that likes that. Is this every client in the world? No, but then it doesn’t have to be. That said, part of Simon and Gus’s point is that too many people burn out early, letting their organisation continue to haemorrhage value because they just can’t face the struggle any more.

Now to the slightly darker part of this posting. The phrase “Potemkin Agile” is a reference to the apocryphal (but untrue) story that Prince Potemkin rigged up fake villages along the Dnieper River to show Catherine II and her court how well his development of the Crimea was going.

As Agile becomes regarded as a good in itself, we should expect to see organisations claim they’ve successfully adopted Agile when the attempt is so half-baked that the result is worse than what came before. I’d include some of Clarke’s horror stories in that category. A long time ago, I went through the Total Quality training at a large corporation. There was good content in the material but most people treated it as a box-ticking exercise to be endured until they could get back to some Real Work, which took the topic right off the agenda. More recently, I’ve seen situations where hit-and-run training has left teams officially “Agile” but lost and miserable; somebody somewhere met their transitioning targets but left out the hard stuff, the follow-up and necessary structural changes. I’ve also seen projects which had all the visible characteristics of an Agile project except that the working code they delivered had no value to the company, because no-one knew what it was for. Think this is just me moaning? Here’s a paper from a respectable business school professor complaining about CIOs who measure value based on a system’s delivery not its use.

Any approach where what people do is misaligned with the organisation is compromised, whether it’s management burning value by only watching costs or technical polishing the wrong code. This begs a huge “How do we get there from here?” question which leaves plenty of room for dissensions like this.

Teaching to aspire to

I know this is the right way to teach, it’s just taking me quite a while to get there.

Stockhausen later said: “In many respects Messiaen did the opposite of what I wanted. He never tried to convince me. That made him a good teacher. He did not give instruction in composition, but showed me how he understood the music of others and how he worked himself.”

via On an Overgrown Path

The HMRC disks: A Silver Lining?

There are some upsides to the recent massive data loss from the UK’s tax authority.

First, even though there was a delay, they did admit the loss and someone senior took the rap. There are too many organisations where this kind of news would never be allowed out.

Second, this breach has been large enough to make the headlines raising, I hope, public awareness of the issues. Of course, the real scandals are systemic rather than this one incident, as Ross Anderson pointed out this was “an accident waiting to happen”. Centralising all the data while casualising the workforce is not safe. If one junior clerk can copy all this data, then we must assume that others already have, but secretly. At the same time, one reason this data is so important is because too many retail institutions have weak procedures when handing out money and the credit agencies are not answerable to the people they rate.

A final benefit is the excellent opportunity for satire. This eBay auction (now withdrawn) has been preserved on The Register.

Update Police have concluded that only one person in the world would want to know the names and addresses of all the boys and girls at around the end of the year. They are now widening their search to the North Pole.

Maybe I should have gone to the APLN Summit

I have an allergic reaction to events called “Summit”, but judging from this posting it sounds like there were some interesting discussions.

Indirectly, I can second the comment on Israel Gat’s presentation, I shepherded a BMC experience report for Agile 2007 and it sounds like a very effective organisational turnaround. The author was so enthusiastic, I couldn’t get him off the phone 🙂

The Balle-Argentee method of business improvement.

A brilliant fable about how good ideas get corrupted. Is your organisation at Phase 8 yet?

via Dave Snowden