My framework for generating great interview problems

If you interview candidates regularly, you would like to have a good pool of questions just to break the sheer monotony of asking the same question over and over again. There is no shortage of posts on the Internet about what coding challenges and questions to ask candidates during various stages of the technical screening process. Instead, I want to share the framework that I use for coming up with interview questions that allow me to capture and evaluate a candidate’s thought process.

Here’s what I do

Every time I solve an interesting problem at work, I try to document my experience and make an attempt to see how I can model the challenge I faced as an interview question.

Let’s take a specific example. Not long ago, I had to implement measures to prevent a location API (which returned metadata about a given latlong) from being used by unscrupulous companies for scraping and re-selling of data. The solution involves a number of architectural as well as algorithmic components, so it allows me to explore the breadth and depth of the solution, depending on the candidate’s experience and areas of strength.

Why this framework works for me

I believe that good interview problems should be centered around issues which you yourself have encountered and solved previously.

You would understand the problem space deeply. This is critical for evaluating the answers, and to ask follow-up questions on peripheral areas.

You would have gone through the complete life cycle of the solution - right from exploring potential approaches, to evaluating the pros and cons of each solution and finally narrowing down and solving the problem. This helps you compare how much the candidate’s thought process is aligned to your own thought process, as you went about solving the problem.

Imaginary puzzles and problems are just that - imaginary. They don’t really test the candidate’s ability to perform in your environment. Picking a problem that you have solved recently would help you get a glimpse of whether the person would be a fit for your organization, solving the kind of problems you solve at work everyday.

Another nice benefit of this method is that you are now simulating a typical work environment where the candidate and you are working together to solve a problem and you can get a sense of how opinionated the candidate is: i.e., if they are flexible and open-minded to consider others’ inputs (the interviewer’s nudging) or if they ignore all external inputs and just chug along with what they think is the best solution. This can surface potential team fit issues.

At the end of the interview, I’ll also tell the candidate that this is one example of the type of problems we solve in the company. This gives the candidate a good overview of the type of work that’s done in the company (so your example better be interesting and challenging!).

There is a gotcha, though

There is only one thing to be cognizant of, though: don’t expect the candidate to arrive at your eventual solution right away, without having the benefit of going through the exploration and evaluation phases that you yourself would have gone through. Remember that this question, being fairly open-ended, is not about giving the perfect answer - it should be a dialog that captures the day-to-day problem solving you would expect somebody to do as a potential colleague.

This post was written by Kishore Nallan