I can’t keep track of the number of articles I’ve read about hiring in the past few years. Inevitably, they all follow the exact same format. First, they claim that hiring is broken. Next, they describe the hiring practices used by XYZ company. The present an extremely thorough analysis of every possible way in which such a practice can be gamed, or produce bad results. Finally, they wrap up by saying that hiring should instead be done on the basis of ABC criteria, only for someone else to write an essay the following week attacking ABC as a broken hiring method.

At this point, I’ve seen almost every single combination of ABC and XYZ. As a fun little exercise, I thought I’d take some time to compile every one of them.

I’m sure lots of people have very passionate opinions about the right and wrong way to hire. Quite frankly, I’ve lost all interest in hearing people’s opinions and anecdotal experiences. Until and unless someone does a rigorous scientific study evaluating different interviewing techniques, preferably using a double-blind randomized trial, there’s no point in beating this dead horse further. Everyone’s hiring practices are broken, and yours aren’t any better.

Look at which University they attended

Pros:

University admissions departments spend an enormous amount of effort and manpower in identifying and attracting the most talented individuals. Why not leverage this investment and expertise for your own company

Top universities generally speaking have top professors, other top students, and challenging course content, all of which will rub off on the candidate

Getting admitted into a top university usually takes a lot of hard work and intelligence. This can be a good proxy for job success

Cons:

There are idiots present in every university. Yes, even at Stanford

University admissions unfairly favors the wealthy and well-connected. By rewarding this, you’re discriminating against the poor, the not-well-connected, and minorities

The skills required for getting into Stanford at 17 (extracurriculars, SAT prep etc) do not correlate to job success as a software developer

How good a student you were at 17, is not very relevant to who you are at 25

Look at which companies they’ve worked at in the past

Pros:

See all the pros listed above, and replace “Stanford” with “Google”

Companies are usually better proxies for the technical skills you need in software developers

Cons:

Those companies have hiring practices that are broken (hence why you’re writing this post). Therefore, you shouldn’t be relying on their hiring process

This doesn’t work for new-college-grads

You’re creating a self-perpetuating loop where someone who got rejected by Google the first time, will never get a 2nd chance ever again

Look at their GPA

Pros:

It’s a very comprehensive metric, which takes into account performance across a large number of courses and exams, over 4 years, covering a mix of homework, projects and exams

It’s a good proxy for how hardworking and dedicated an individual is

It’s a reasonable proxy for how smart an individual is

Cons:

Ask them for references

Pros:

Gives a view of their actual on-the-job performance

Cons:

Do back-channel references

Pros:

Gives an unbiased picture of actual on-the-job performance

Cons:

You will be considered evil for doing this

Hard to consistently find the right people who can provide accurate references

Rewards people with great social/networking skills

Punishes people who’ve worked with bad/abusive managers in the past

Take home projects

Pros:

Cons:

This is basically unpaid work. You will be considered evil for doing this

Very time consuming. Discriminates against people with lives and families

Extremely easy to game. People can get their friends to help them. People can spend 30 hours doing something which should be a 10 hour exercise, just to look really good

Audition for the role as a contractor

Pros:

Actual on-the-job work

See what they are like to work with

See their actual work product

Cons:

Depending on their current employment agreement and immigration status, this could be illegal

You’re asking them to take a week of holiday, just to work for you. People will hate you for doing this

Discriminates against people who have lots of companies showing interest. They can only afford to do this audition at 1 or 2 places

Discriminates against people with lives and families

Ask them to describe their past projects

Pros:

If they’ve worked on challenging, relevant projects in the past, they can likely be successful at your company as well

Building a successful product can be a reflection of talent

Cons:

Ask for source code from significant projects

Pros:

Actual work product, that’s a good reflection of their capabilities

Cons:

Most work product is confidential. Google will never allow you to show your work-source-code

Not everyone has the time to work on open-source projects. Punishes people with families

Not everyone is interested in spending their free time on open-source projects. Punishes people with diverse interests

Quiz them on their knowledge of various technologies, tools and frameworks

Pros:

If they know about, and how to use technology X, they will be able to use X to successfully fulfill their job role

Shows that they care enough to research this stuff

Cons:

Brain teasers

Pros:

Tests their overall intelligence and problem solving skills

Cons:

Either you know it or you don’t. Relies on you being lucky enough to get a flash of inspiration, or you having heard it before

Not directly related to programming

Even Google says it is useless

Live Coding Exercises

Pros:

Live test and demonstration of coding skills, that’s not susceptible to the above cons

The fact that FizzBuzz is so effective, demonstrates the need for more in-depth live coding tests

Completely fair and impartial. If you solve the problem, you’re scored highly. If you don’t, you’re scored poorly. Such meritocracy

Cons:

Pair programming on a “real” problem

Pros:

You’re working on a real problem, that resembles the day-to-day responsibilities

There is no “known-right” answer. Collaborative exploration

Cons:

It takes developers weeks to get ramped up on a specific code-base, domain expertise, and technology stack. Expecting someone to drop in one afternoon and start making an impact, is not very realistic

Highly variable. Depending on the day they come in, some candidates can be given very easy issues to work on, involving languages/domains/technologies they’re already familiar with. And others can have the complete opposite experience.

In order to assess someone’s potential, you have to give them problems that push them to the limit, even if it isn’t “realistic”. There’s a reason Intel CPUs are evaluated using synthetic benchmarks, and not MS Word

If you gave every candidate the same “real issue”, of challenging but not insurmountable difficulty, in a sandboxed environment, requiring minimal prior domain-knowledge, with flexibility on language/technology-choice: Congrats, you’ve reinvented live coding exercises.

The German translation of this essay has been published in golem.de.

And a Russian translation can be found here on habr.com.

Discussion thread on Hacker News.

A variant of this essay was earlier published here.