If you’re a regular reader of this blog, you know that I’ve come to rely quite heavily on data. I’ve counted typos on resumes, I’ve sifted through a corpus of engineering offers, and I’ve skimmed thousands of recruiting messages to tag them by personalization level. This post, however, is going to be a bit of a departure. Rather than making broad, sweeping conclusions based on a lot of data points, I’m going to narrow in on one story that happened, in part, because of some data I gathered during an experiment. I think it’s a really cool story, and I can only hope that there will be more stories like it that, in time, will enable me to write another post with lots of graphs.

The experiment in question was thus. Last fall, I showed a set of anonymized engineering resumes to about 150 engineers, recruiters, and hiring managers and asked one question: Would you interview this candidate? It turned out that not only did both recruiters and engineers largely fail at predicting who the strong candidates were, but, much more importantly, no one could even agree on what a strong candidate looked like in the first place.

These results were quite startling, and they left me scratching my head about the implications. After all, resumes are such a huge part of how hiring is done. Of course, it wasn’t always this way. From Chinese civil servant exams to Thomas Edison’s somewhat infamous general knowledge tests, filtering for positions requiring specialized knowledge was, up until the last century, accomplished largely through a variety of aptitude tests. Even as the resume rose in prominence and became a ubiquitous arbiter of potential fit, its format has gone through a number of evolutions. Things that were commonly included on resumes at various times (and still are in certain cultures) — a photo, marital status, age, religion, height, weight, blood type, and political affiliation — are no longer en vogue, and it’s certainly not inconceivable that, in the future, asking for a resume will seem just as silly as these now-outdated practices.

So, if resumes aren’t a good signal for hiring engineers, what is? In my blog, I wondered if, instead of a resume, it might be interesting to have people write a bit about something they built that they were excited about. I was excited, in turn, when my friends at KeepSafe actually decided to try this out — for a month, candidates who applied through KeepSafe’s No Resume campaign were evaluated purely on what they wrote about their projects, and this writing sample would be the only thing used to decide whether someone would get an interview. This is the story of what happened.

KeepSafe, in some ways, came to this experiment out of necessity. Like many other small startups, they were feeling the acute pain of hiring engineers in this market. They had a product people loved, a ridiculous user to engineer ratio (39M users for 6 engineers), and the requisite hip South Park office. Along with that, though, they had quite a high hiring bar and consistently found themselves competing for talent with the likes of Google. And more often than not, they’d lose.

To stay in the game, KeepSafe needed to change up the rules and tap into a different pool of people. The hope was that out there were plenty of talented engineers who were getting overlooked because of their lack of resume pedigree but who were passionate and skilled and who would be awesome hires, given the chance to show what they could do.

KeepSafe’s experiment struck a chord, and in the first two days, over 400 people submitted descriptions of stuff they’d built. Awesome stuff like this.

Submissions varied in length. Some were just a sentence. Some were multiple pages and supplemented with links to demos. Not all submissions were awesome. Some were generic, copy-pasted cover letters espousing their interest in the software development life cycle, and some were just disconnected links. Those KeepSafe cut immediately. The rest were a lot tougher to cull.

Though I didn’t participate in the judging, one thing that struck me about the submissions I read was that my normal approach was useless here. Normally, when I look at resumes, I can make a yes/no decision based on proxies like past employers within 10 seconds. With KeepSafe’s submissions, I couldn’t rely on proxies at all. With each person, I had to think about what they built, imagine it, understand it. In addition, I found myself starting to getting a real read on people’s interests and trying to imagine what projects they might want to tackle at KeepSafe, something that’s much harder to gauge with the traditional resume format.

All of this was weird and slow, but it felt really good. And then I realized that unless your achievements fit into a convenient mold, you’ll lose the best parts of you when you try to beat your windy path into a nice clean line. Resumes don’t have an explicit section for building rockets or Minecraft servers, and even if you stick it somewhere in “personal projects”, that’s not where the reader’s eye will go. The sad truth is that if the reader doesn’t like or recognize what they see before they get to the rockets, they will likely never get there at all.

***

By traditional Silicon Valley standards, AJ Alt didn’t look especially good on paper. He hadn’t attended a brand-name school, his GPA wasn’t high enough to stand out, and his only professional programming experience was a multiyear cybersecurity stint at a huge defense contractor, the nature of which he was forbidden from discussing, on a resume or otherwise. His GitHub, full of projects spanning everything from a Python SHA-1 implementation to a tongue-in-cheek “What should I call my bro?” bromanteau generator, hinted at a different story, but most people never got there. While AJ’s government work experience gave him a good amount of cred in the public sector, he found that making the move to industry, and startups especially, was near impossible. It wasn’t that he was blowing interviews. He just couldn’t get through the filter in the first place.

Of the 415 people who applied to KeepSafe through the No Resume campaign, 18 ended up interviewing, and 5 came in for an onsite day of coding. One received an offer. It was AJ, a candidate that Zouhair Belkoura, KeepSafe’s cofounder and CEO, readily admits he would have overlooked, had he come in through traditional channels. Since starting a couple of months ago, AJ has built out an open source Android animation library and was one of the co-creators of a brand new security feature within the app.

When I asked Zouhair if KeepSafe would consider the experiment a success, his answer was a resounding yes. According to Zouhair, “The overall quality of applications seemed to be a lot higher than through our normal resume channels. We also met more people who seemed to genuinely like programming and wanted to talk about it [rather than people just applying because they wanted a job].”

The team has decided that they will continue to hire without resumes for the foreseeable future — the funnel numbers for this process weren’t too different than what you’d see in a more traditional setting, and, with time, as the team gets better at making value judgments based on writing and projects, they will only improve. Resumes work well for companies like Google because their strong brand drives a revolving door of inbound applicants. Copying that process, if you’re a smaller brand, however, can be detrimental. Of course, this experiment had a small sample size, and one hire is more anecdote than gospel, but if replacing the resume with a writing sample is a good way to get at a different, yet highly skilled and engaged talent pool, then it’s worth a shot. As Zouhair told me, “Our culture is that code speaks louder than credentials, and now we have a hiring process that reflects that. [When we made the hire], we had never even seen AJ’s resume. And we don’t need to.”

Interested in applying to KeepSafe without a resume? They are still accepting not-resumes at https://www.getkeepsafe.com/noresume.html. Think resumes are generally a dumb way to filter for engineering ability? Check out what I’m working on now, interviewing.io.