Stephen Freeman Rotating Header Image

Doing pair programming tests right

In her rant on the state of the industry, Liz Keogh mentioned coding in the interview, which triggered several comments and a post from Rob Bowley, who reminded us of Ivan Moore’s excellent post. I think actually typing on a computer is essential which is why I’ve been doing it for ten years (enough with whiteboard coding), but I’ve also seen examples of cargo cult code interviews where the team didn’t quite get the point:

It’s a senior responsibility
Pair programming tests should be conducted by senior developers. First, this shows that the team thinks that actual coding is important enough that senior people have to get involved, it’s not just something they delegate. Second, now matter how smart, juniors will not have seen many different approaches, so they’re more likely to dismiss alternatives (technical and human) as bad style. They just don’t have the history. There are times when a tight group of young guns is just what you need, but not always.
Do it together
Be present for the work. Don’t just send the candidate off and tell them to submit a solution, the discussion is what’s important. Otherwise, it turns into a measure of how well someone can read a specification. It also suggests that you think that your time is too valuable to actually work with a candidate, which is not attractive. And, please, don’t play the “intentionally vague” specification game, which translates to “Can you guess what I’m thinking?” (unless you’re interviewing Derren Brown)

Be ready
Have your exercise ready. Your candidate has probably taken a day off work, so the least you can do is not waste their time (and, by implication, yours). Picking the next item off the backlog is fine, as long as it doesn’t turn out to be a configuration bug or to have already been fixed. One alternative is a canned example, which has the benefit of being consistent across candidates. An example that is too simple, however, is a good primary filter but limits what you can learn about the candidate, such as larger-scale design skills.
Have a proper setup
Your netbook is cute, portable, and looks great. That doesn’t make it suitable for pairing, not least because some candidates might have visibility issues and the keyboard will have keys in the wrong places. Use a proper workstation with a good monitor so you can both see, and talk about, the code
Allow enough time
Sometimes things take a while to settle. People need to relax in and you need time to get over your initial flash response to the candidate. Most of us do not need developers who can perform well under stress. I’ve seen great candidates that only opened up after 30 minutes. You also need to work on an example that’s interesting enough to have alternatives, which takes time. If you’re worried about wasting effort on obvious misfits, then stage the exercise so you can break early. You’re going to work with a successful candidate for some time, so it’s not worth skimping.
Give something back
This is something that Ivan mentioned. No matter how unsuitable, your candidate spent time and possibly money to come to see you, and deserves more than a cup of tea. Try to show them something new as a return. If you can’t do that then either you don’t know enough to be interviewing (remember, it should be a senior) or you messed up the selection criteria which means you’re not ready.


  1. David Harvey says:

    Nice. Three things:

    1) Allow enough time. You can’t do a reasonable pairing interview in an hour
    2) Pair the candidate developer with more than one person. In particular, if the first pair sets up the problem and the pair write some tests, you can learn a lot from the way the candidate explains where they’d got to when he pairs with a second partner
    3) Understand the difference petween a pairing interview and a coding audition. They’re not the same.

  2. Can I also add to be ready that if you are using a stock example make sure everything is setup properly.

    I’ve had several experiences, the most memorable was discovering that the database server hadn’t been configured correctly and having to spend the lion’s share of time sorting their machine out.

    That really should have been an indication to walk out, but you live and learn.

  3. Johnno Nolan says:

    I like to do two stages of tech interview. Firstly an off line technical exercice where I can analyse a design approach then I do very simple excercise in the interview. This shows 2 things, how well can people follow a specification and a very simple example of solution design. It also provides a subject of conversation in the face to face which demonstrates ability to critictially anlayse and communicate their solutions. After this I spend 20 mins doing a software kata. This shows ability on the keyboard, as well as how they work in a pair.

    Its a big ask for developers but that additionally is a sign of commitment.

  4. Johnno Nolan says:

    Having said its a ‘big ask’ – that’s the feedback i recieve. Why as a developer would you have a problem prooving your developement skills?

  5. Mark Piper says:

    Generally I find a barrier in these things is explaining the domain to the interviewee, and giving them enough background to get going. Recently I’ve been wondering whether it would be better to prepare candidates to bring along their own code to work on. That ought to a) demonstrate that they’ve actually written some code before, outside of what they’re required to do for their job, b) make sure “I didn’t understand the problem” isn’t an excuse, c) give some insight into their real coding style, and d) given that the other person in the pair won’t have seen the code before, give a better insight into “real” pairing.

  6. @Mark an interesting idea. It depends what you want to test for and what experience you require. The advantages of working with the system code are that you can see how quickly they pick things up and the candidate won’t be unpleasantly surprised if they take the job πŸ™‚

  7. Mark Piper says:

    Well, if they turn up with PHP, interview over πŸ˜€

  8. […] Great advise for using pair programming as a job interview tool. TL;DR treat job candidates with respect. Be present for the work. Don’t just send the candidate […]

  9. Grzegorz says:

    I agree 100% that proper coding interviews are the best. However, I still believe that whiteboard type of coding exercises can reveal more than a phone interview, in particular:
    (i) if the candidate actually used the technology he claims in his CV (ii) if the he/she has any idea about algorithm complexity and can efficiently solve simple algorithmic problems.

  10. Rob Fletcher says:

    We will play the ‘intentionally vague specification game’ with the aim of seeing whether the candidate is the kind of person who will seek clarification and ask questions or just make assumptions & gloss over ambiguity & blaze ahead implementing what they think the spec means.

  11. @Rob Intentionally vague is a good idea provided it goes with a face to face discussion. What doesn’t work is sending someone off with a vague specification to see if they agree with you.

  12. Arlo Belshee says:

    I did a lot of pair interviews in a previous life. Still believe in them; just am not looking for coders on my team right now. We always looked for several things. For each thing, we picked how we would assess.

    In teams that pair continuously, I have found skills to be uncorrelated with success. The pairing creates an environment in which everyone can learn, and quickly. So I don’t care what skills they have. I care how good they are at taking advantage of learning opportunities. People with one skill will, six months later, have one specialty. People who know how to learn and teach will, six months later, have 7 specialties each as deep as that other candidate’s one.

    This drives me towards a lot of behavioral interviewing approaches. I identify the traits we fundamentally need – degree and quantity – on the team. I then build a team (one interview at a time) that fills in our missing traits. And at some point, the ability to code in a pair comes up. But not because we are looking for a SQL master. Because we are looking for a [verbal communicator; problem framer; analytical problem solver; innovator-creative; integrator of others’ ideas (pick one)]. We want someone who not just codes well in a pair, but brings the right capabilities to our team.

    And, when we are interviewing devs, we find there are 2 kinds: those who can talk the talk and those who can do it. These are not independent sets: some people can do both and lots of people can do neither. So we use a story-based behavioral interview to see if people have it, and a pair experience to see if people have it. People will generally fail whichever one is non-native to them, and then show their true capabilities in the other.

    I never just pair for an interview. I never just interview for an interview. I never ask puzzle questions. I never make anything intentionally obscure or high-pressure. I care about how this person will fundamentally transform my team. The only way to measure that is to tell them that is what I’m looking for and give them every opportunity to do it without stress. Then assess the results.

    And find a good way to compare the person that does it well in a pair with the one that does it well in stories. Because both are strong indicators if you know how to assess, and either could be your better candidate.

Leave a Reply

Your email address will not be published. Required fields are marked *