I've had a couple of preliminary discussions with a startup regarding a position with their company. Two short conversations with recruiters. And now they've asked me to do an online "coding exercise" that will take 2 hours. First of all, we're not far enough into this process for them to ask me to spend 2 hours on a task. I have not yet spoken to anyone I would be working with or for. I do not yet know if I have a strong enough interest in this company to spend 2 hours on a "coding exercise." But that's not even the point of this post. The point has to do with the content of the exercise. I took a look at it and quickly realized that it consisted of two programming puzzles. I didn't do the puzzles. Instead, I sent this email to the recruiter:

I took a look at the programming puzzles and am not going to participate in that process. If y’all choose to pass on me over this, I understand. But trust me, I have good reason for rejecting these types of puzzles. The industry as a whole needs to get off this track. Here in the Bay Area, these types of puzzles are still fairly common, although they are being abandoned by an increasing number of organizations (including some large household name tech companies who used to be big proponents of these puzzles). They’re not effective. They do not contribute to good hiring decisions.

I’ve conducted literally hundreds of tech interviews, and I’ve never asked a candidate to do puzzles like these. I’ve had other people participating in the interviews with me who did have the candidates do puzzles, and I learned from experience that they add no value. At best, the puzzles only indicate how good the candidate is at solving programming puzzles.

First of all, the puzzles are academic and are really only of any value when interviewing recent comp sci grads who have little to no real work experience to discuss. So in an effort to ask them about something they know, we ask the academic questions. More importantly, they’re off target. They’re not representative of the work we actually do on a daily basis. The fact that there are books and websites to learn how to solve these puzzles in order to improve the chances of getting an offer should be an indication to anyone paying attention that the puzzles are off target. Solving programming puzzles is its own skill. I’d say there is little to no correlation between being good at solving programming puzzles and being good at writing real world software. If these puzzles are off target, then what is on target? A discussion about code they've written, architecture, best practices, team dynamics, user experience, and OO design (which is not the same thing as OO programming) is what gives me a good understanding of a candidate’s capabilities. The puzzles are time consuming. I’m a busy person who can’t afford to spend 2 hours on a task that I believe has no value. When I’m the one doing the hiring, I have numerous other ways to spend those two hours that will help me make a good hiring decision.

Evaluating candidates based on their ability to solve programming puzzles seems like an attempt to turn recruiting into a formula. Note that I have done recruiting myself and have managed recruiting teams. In my experience, effective recruiting is never formulaic. Recruiting is far more art than science. Furthermore, taking the formulaic approach and then attempting to automate it with an online tool is WAY off track. After all, this should be people recruiting people. If an employer wants to find out what my skills are, they need to talk to me.

I welcome the opportunity to discuss this with you or other members of the hiring team.

What do I expect to happen now? I expect them to pass on me. But I'm okay with that. I'm so confident of the ineffectiveness of programming puzzles as a way to measure a programmer's skill that I'm leery of any organization that uses these puzzles as a predictor of a candidate's abilities.