Stephen Freeman Rotating Header Image

Software culture

Do do XP

In this post, Tobias Mayer argues against doing Extreme Programming (XP). I have a lot of time for Tobias, but I think he’s wrong on this one. I don’t know who he’s been talking to, but some of this is “strawman” argument, and I’d be more likely to be convinced if Tobias had tried XP just the once. XP is not a universal solution, but it is one possible choice and we know how to make it work.

As an occasional XP advocate, I don’t “blame Scrum for the lack of good development practices in the software industry”, I blame the software industry. If we worked in an effective industry, we wouldn’t be having methodology wars because things would just work. Now this same industry is messing up Scrum too by just taking on its ceremonial aspects. On the other hand, to blame XP for blocking good practice is just bizarre.

XP is a tiny movement that attracted some attention. What XP (version 1) did achieve was to show that it is possible to break through the logjam of cautious procrastination that still cripples many development teams, but without resorting to hackery. It gave teams a reliable package of practices that just worked. Of course XP didn’t take over the world because it’s not suitable for everyone–not least because it requires a degree of focus and skill that is not appropriate for many teams. Kent Beck’s presentation of XP version 1 was extreme on purpose: it was designed to inspire us huddled masses, and to stretch the boundaries of what was considered possible in software development, to reframe the discussion.

I think Tobias has forgotten just how far we’ve come in the last decade. That we have a craft movement at all is because XP put the actual writing of code back into the centre of the discussion–just look at who’s involved, it’s the same people. He also forgets just how counter-intuitive many of the XP practices are, especially compared to the direction the industry was moving at the time.

Tobias writes that the good development practices were spreading slowly at the time, but I’d argue that without XP we’d still be waiting. Test-Driven Development is still not that widely accepted and even the original C3 team didn’t adopt it fully until Kent was writing his book. Refactoring had a small academic following, but it’s not very safe without the compensating practice of TDD. I suspect most teams still ban changing code unless it’s to change a feature. Pair programming is still a very hard sell and, again, works much better in the context of TDD. I’ve seen enough Scrum teams that have not found a coherent set of technical practices. To say that they just need to improve their Scrum implementation begs the question of how Scrum is adopted and the limits of self-organisation.

Some final nit-picks. There are two editions of the XP book, the second is more recent than 12 years and has a “softer” approach to the methodology. As for the relevancy of the practices, the C3 project worked in an environment (Smalltalk/Gemstone) that still outclasses what most of us use today. Much of the work in the XP community has been to try to recreate that flexibility in inadequate current technical environments. What’s really scary is how slowly this industry moves.

Software Nightmares (2)

About a month ago, Michael Feathers wrote another post that cited Gordon Ramsey. That reminded me to follow up my earlier post with observations from a couple of episodes from UK series 4 that make the point (thanks Channel 4).

First up is a curry restaurant in Nottingham (which has no shortage of competition) opened by an ex-sales manager and glossed up to look like an ’80s night club. Being from sales, his view is that the customers gets what they ask for, so he’s offering design-your-own curries—whatever combination you want. The result is that the kitchen brigade (who are good) can’t cope with the variety and turn everything to mush. When Ramsey has them cook all hundred-odd dishes on the menu, the waiting staff can’t tell which is which.

The owner sees the process as simple order-taking. Offer the customers whatever they ask for, regardless of the effects on the demoralised staff and low quality. It turns out that the customers aren’t curry experts and don’t like what they’re getting, so the restaurant is losing money. Ramsey’s solution is to cut the menu back to something the brigade can do well, and to offer what the customers what they actually want.

Second up is a carvery outside London that’s been bought by Scott, an ex-IT consultant. It’s losing money so, in an attempt to fill the place, he’s offering two-for-ones at below cost price; the only people who put up with the dreadful food are pensioners. He’s hired a cheap brigade who can’t even cope with the grill idea that Ramsey proposes, and the kitchen is so decrepit that Ramsey has it condemned. Scott is discovering that reducing costs isn’t working and risks poisoning the clientele.

Scott seems most comfortable in the office behind a computer, presumably planning the work that everyone will do. He appears to be the only owner in any of the series that has no interest in food or customers—the value part of the equation. He has no idea if his people are competent or what constitutes good service.

Back-to-back, the episodes make a nice pair. To belabour the point, we can’t do good, valuable work when the process is one-sided: when requirements are forced through an organisation without thought for the consequences, or when work is driven from behind the scenes without enough attention to a paying customer.

It should be obvious, but we need reminding. Hands up if you work in one of these places.

Software Nightmares

This post has been stewing for a little while, and now I’ve been kicked into writing it up by Naresh Jain’s post on Lessons Learnt from Restaurant Business.

Since Channel 4 in the UK started supporting the Mac for their online replays, I got hooked on Gordon Ramsey’s Kitchen Nightmares series1. I’ve never worked in a restaurant and I know that TV programmes are a manufactured narrative, but (inbetween all the swearing) there are some interesting themes that come through every week:

  • cook stuff your customers actually want. Some owners get carried away with bigger ideas than they can handle;
  • reduce the menu to something your staff can cope with. Bloated menus and fancy presentations cripple the kitchen staff. Cut it all back to something they can do well;
  • help the staff take pride in their work. Producing stuff badly for unhappy customers is wasting people’s lives, and probably not sustainable. One of Ramsey’s triggers for cussing out the brigade is when they’re not trying—and watch their reactions when they turn it around and he praises them;
  • cook your own stuff. Don’t rely on outside prepared stuff unless there’s a very good reason2. Bring in good ingredients and, um, cook them;
  • take responsibility. You’re the chef/owner/manager/waiter, so do your f***ing job;
  • communicate. Head chefs should be talking all the time so their brigade knows what’s going on, waiters and chefs should discuss the tickets so they know what’s ordered, waiters and managers should talk so they know what’s going on with their customers; and,
  • see it as it is. The building looks run down, the kitchen is filthy, the food is disgusting. Stop fooling yourselves.

I know it’s relatively easy to come in and see how to rescue a business that’s months away from failing, after all an outsider has the advantage of not having been sucked into the mess. But, even through the box of mirrors that is TV, Ramsey shows two strong drives: total focus on providing the best service to the customer, no excuses; and, total respect for the craft of cooking. He just lights up when he finds a junior with ability and enthusiasm.

Somehow, I feel this ties in with a post from another coach given to immoderate use of language, Mike Hill wrote about how raising your internal quality makes you go faster. I think there’s a commonality of purpose there that we should take note of.

1) I mean the UK version of this series. The US series is like a boil-in-a-bag version: manufactured, over-spiced, unsatisfying. In fact, the opposite of everything Ramsey is promoting. But that’s another story.

2) Yes, I know there have been some “scandals” with Ramsey’s London restaurants, but that doesn’t invalidate the point.

Why project automation

Via Brian Marick’s other site:

“Machinery helps you pay attention to what’s important,” Wajswol says. “In cheese making, there are a couple of things you need to focus on. If you can eliminate the nonsense—the mundane, nonskilled steps, like feeding the animals or warming milk correctly—you can spend more time focusing on the texture of the curd and making sure the product comes out good.”

(my italics)

"Broken gets fixed; shoddy is forever"

…which is why maintaining quality is so important.

From Hillary Johnson at Agile Open Northwest

"He doesn't mean that about Scrum"

In very bad taste, but very funny: “Hitler’s Nightly build fails”.

and we should remember that Stalin won…

Experienced Agilista's proved wrong (again)

So, Jurgen Appelo is unhappy that some of the more experienced Agile names have been telling him what to do. In particular, apparently they’ve been doing so without understanding complexity theory; he’s not reacting well.

In between the ranting, much of what Jurgen says is obviously true. For disorganised teams, adopting Scrum and nothing else will help them get more organised and productive, as seems to be his case. But he then goes on to say that anyone who tries to clip his wings and tell him how to develop software cannot be agile because they’re not adaptive enough—and by the way, he knows lots of stuff about complex adaptive systems that other people don’t.

My first reaction is to suggest that he not underestimate some of the people he’s reacting to. Ron Jeffries can certainly be provocative, but I don’t think he does it to try to convince people he’s smart. And some of us have been investigating complex adaptive systems for years.

What I think Jurgen is missing (or at least not making explicit) is that there isn’t a single axis between chaos and ordered, some aspects of an organisation do need to be ordered (in the end, we’re dealing with physical machines here) and some things complex (the people bits, usually). I may use complex adaptive techniques to understand what to build and how to communicate that, but I also want the thing to work reliably and not to be dragged down by a fear of making changes.

Sure, people in the community say dumb things, but we have actually learned some things over the last ten years. One is that we see team after team that has hit the wall because they didn’t work cleanly. Some have the luxury of a financial buffer which allows them to continue working sub-optimally. Just a few teams have understood the real trade-offs and can undercut the opposition by delivering faster and more reliably—and no-one promised that it would be easy or quick to achieve.

Lean and Agile: should cousins marry?

Dave West has written a cautionary posting (Lean and Agile: Marriage Made in Heaven or Oxymoron?) on the dangers of taking a simplistic view of Lean and Agile. He’s right that a naive reading of a Lean approach to software will just trap us in another metaphor, manufacturing, that’s as inappropriate (or appropriate) as we once thought building construction was. And yes, there will be teams that adopt the techniques without understanding the meaning behind them, just as we’ve seen with all the Agile methodologies. I particularly like his reminder to show the bigger picture by making visible the current product backlog, with all its inconsistencies and misunderstandings.

I think there is a place for the production features of Lean in software practice, such as the idea that when I get something working, it really works and it stays working. I think there’s value within development on staying focussed on the task at hand and getting in “Done, done”. For some teams, just getting their system to a state where they can safely make and deploy changes is a huge advance. As Martin Fowler points out, there are quite a few “Agile” teams that can’t achieve that. Working in a reliable code base where all the mechanical stuff just works is so much more productive than the alternative, which counts, I think, as removing waste.

David Anderson claims that nowadays the most significant waste is usually around development rather than within it. It’s in the weakness of the organisation’s communication and prioritisation, the stuff that Dave has pointed out we need to support; teams waste effort when they don’t communicate and understand.

Once I started exploring deeper into the Lean material, I discovered the Toyota Product Development System which some think is even more important than their Production system. From a Production point of view, it looks very wasteful because they work on multiple possible solutions but choose only one, which allows them always to deliver on time. From a Product point of view it’s worth the cost because slipping the date is too expensive, and it’s not totally waste because they record what they’ve learned from the failures and use it the next generation of products. I think this is where much of the activity that Dave wants to remind us about belongs, and many software teams just don’t spend enough on exploring the range of features and technical issues available to them.

To mangle Dave’s metaphor, I don’t think we should expect wedding bells because the couple is already too closely related. There will be mistakes of interpretation in the field, such as dismissing retrospectives as waste, but, properly understood, the underlying values share too much DNA.

SOLID Development Principles – In Motivational Pictures

Excellent series of images from Derick Bailey. Here’s an example:

Single Responsibility Principle

Just because you can, doesn’t mean you should.


RT @Jtf

Some Things l Need to Know about Programming I Learned In Music College

In a previous life I took a music degree in Bass Trombone, which is a discipline that’s even more geeky and with a worse gender balance than Software (see this meeting if you don’t believe that’s possible). Over the course of the degree, I raised my bar from Enthusiastic to Not-too-embarrasing-to-appear-in-public. In the spirit of this famous article on Pair Programming, here are a few things I learned there that (if I squint) seem to apply to software.

Quality follows a power law Every time I practiced and got to play with better ensembles, the next level up was an order of magnitude improvement, not just linear. I could imagine being as good as the players one level above me, but two levels was a huge jump. The standard of serious professionals nowadays is just astonishing. As an example, one of our ensembles worked on an Edgar Varèse (obscure but influential modernist composer) piece. For the première in the 1920s, Varèse complained bitterly that he only had 100 rehearsals, we did it in about 10 rehearsals, our conductor performed it with a group of freelancers after working on it for an afternoon.

Quality is fractal To put it another way, good performances requires attention to detail at all levels: from the conductor’s management of the overall structure, to players’ phrasing of their individual lines. The better the ensemble, the more levels just work. It was quite a shift for me when I started joining groups that played in tune, a whole area of insecurity just disappeared and I could now use the effort I’d spent on compensating for inaccurate tuning for something more important.

Play for the audience They’ve paid to hear you. They don’t want to hear your technique, they want to hear music. One of my teachers liked to point out that anyone can make great music work, but good players can make bad music sound better. At the other end of the scale, I once heard a scaled-down version of The Rite of Spring where the Bass Trombonist dominated the orchestra; as a practitioner, I was impressed but it was ugly and self-indulgent.

Don’t take it up unless you mean it The perfoming arts are tremendously rewarding if that’s what you want do. If not, it’s a hard trade involving lots of stress and effort—and there’s a limit to how long you can stand discussing mouthpiece diameters. I met several older professionals who hated the business but had nowhere else to go, and the story goes that one of the trombonists in Toscanini’s NBC orchestra wrapped his instrument around his music stand at the end of his last day before retirement.

The section takes the blame (and credit) together Brass playing, particularly at the low frequency end, is a collaborative activity. You spend much of your time playing chords, so no-one can tell that you’re better than the rest of the section. Your best hope is to try to raise everyone’s standard. When it works, it’s just fantastic.

Some people’s abilities are just obvious I met a few players who were clearly here on Earth to play their instrument, that’s just who they were. These are the kind of people who get top-rank jobs before they’ve finished college. Our Head of Brass had been a child virtuoso, he didn’t understand why people played wrong notes. Why would they want to do that? At the other end, there were some who should have had their instruments confiscated.

Sometimes people take a while to shine Other players are not so obvious. I was lucky enough to meet Ed Anderson (Cleveland Orchestra) who was one of the best. He told me that at college, he’d played in the opera orchestra because he didn’t get enough sessions with the (higher prestige) concert orchestra.

Reality, deal with it In a performing discipline there is nowhere to hide, everyone knows how good you are all the time. I had a bit of a crisis in my second year when a new Tuba student took the time to point out my shortcomings; the message wasn’t pleasant to hear but it worked, I was much better by the end of the year. Later a one-off lesson with Arnold Jacobs, who spent a lifetime researching the mechanics of wind playing, changed my playing life because he could show me what was happening to my breathing and how it needed fixing.

How do you get to Carnegie Hall? Practice! It turns out that this topic has just come up on Tim O”Reilly’s blog.

What’s the dynamic range of a bass trombone? On or off