Tests and assessments are no longer an unusual part of the hiring process. But for the senior-most executives? For C-Suite roles? Executive recruiter Korn Ferry recently introduced a new system, called KF4D, to help them place better bets on talent at the top of the ladder. It’s just another way we’re all going to have to get more accustomed to collaborating with algorithms.

What follows is an edited version of my conversation with Michael Distefano, Senior Vice President and Chief Marketing Officer, and Dana Landis, Vice President of Global Talent Assessment & Analytics.

HBR: Let’s just start with a quick description of how this particular tool works.

Distefano: Let’s say [a CEO] needs a new chief of staff. He’d ask people to take a quick assessment in KF4D. We’d ask the company to tell us, “What does this chief need to be competent in?” You could choose 15 skills out of 38, and we’d ask you to sort them into high, medium, or low [priority]. And then tell us about your [company] culture. Finally, what does the person need to do – e.g., are you hiring them to drive change? We take that input from the client, and combine with a deep catalogue of benchmarks. The algorithm takes that together, crunches it, and comes back to us with the profile of the person we are looking for. Our recruiters go out, find candidates, and then they take an assessment.

HBR: How do you get that catalogue of benchmarks — that big data-set – to begin with?

Distefano: Over the last number of years we’ve made numerous acquisitions in talent development and leadership development, and [that led us to] acquire more datasets and databases — almost 50 years worth of data for executive levels. So we were able to pull these together into one data warehouse [containing] over two million assessments.

Landis:We now have enough data to start answering questions like, “What’s differentiating leaders in different contexts and cultures and jobs?” When you pull all the data together, you start to see the patterns. It’s fascinating to know what doesn’t change, and what does change across regions and levels.

Insight Center The Future of Collaboration Sponsored by Accenture How tools are changing the way we manage, learn, and get things done.

What are some of the things that are different for C-suite roles, as opposed to roles that are lower on the ladder?

Landis: One example is conscientiousness. At the bottom it’s critical, but it’s a lot less so at the top.

Distefano: [Conversely] agility – lifelong learners with a high tolerance for risk – that’s kind of a universal attribute.

Is it unusual to have data on senior executives? It seems like these kinds of tools are more often used at lower levels.

Landis: Data on global and very high-level populations is rather hard to access. We usually see these [assessment] tools at mid-level and below.

Some leadership norms are very specific to a certain company culture, or country culture. How does an algorithm take that variability into account?

Distefano: Because we’ve got [a database of] over two million assessed executives, we can cut that so that we can see how a CFO at a state-owned firm in China is different from a CFO in private equity-owned firm in Silicon Valley.

I find it interesting that you focus so much on measuring culture and fit, two things that seem like they’d be really tough to quantify. Why not leave that to the interview?

Distefano: I like to say, “You get hired for what you know, and fired for who you are.” Over time, our findings were that of, say, CFO candidates, everyone can add, everyone knows the regulations and tax code and so on, but there’s two places that go awry. One is you’re not a good cultural fit. Second, we all have things within our psyche that derail us as we go up the ladder.

Landis: What the assessment does is pick up high and low scores and develops an interview guide off of those scores. The assessment works as a guide – here is where to probe further, here’s where there might be a problem.

Is it all based on self-assessment? If so, how do you know people are answering honestly, with self-awareness?

Landis: The way it’s designed, it’s nearly impossible to do the social desirability thing. You’re not allowed to say you’re great at everything — you’re forced to prioritize. The questions are more behaviorally anchored. It’s not questions like, “Do you like to hang around with people? Are you a team player?”

Distefano: Oftentimes we’ll ask you, “Of these three things, what are you the best at?” And maybe you could choose between something like, strategic thinker, communicator, or global perspective. After you pick one, then we ask, “Of the remaining two what are you best at?” By process of elimination you’ve told us what you’re not best at. We’ll ask you again, mixing [those skills] in against other things, and see if a pattern emerges. What’s your top strength? And if you’re changing your answers, that’s telling us something else too. If you have one dominant strength, you’re a professional. If you have many, maybe you’re high potential.

Landis: You could have [a problem with] someone very un-self-aware. No assessment can account for that.

It sounds like the size of the dataset is really an asset. But is more data always better?

Landis: There tends to be this kitchen-sink approach in measurement. You find HR departments that are measuring every conceivable thing, piling tool on top of tool to not miss anything. You get this sense of exhaustion: “We’re measuring everyone and everything — we’ve got all this weird data and we don’t know what it’s telling us.” One thing we realized we could do is clarify what matters and where [our clients] are wasting their time. For example, heavy IQ testing. Anyone at that [very senior] level is already smart enough. That’s not the problem. We have to pull their attention to measuring those things that actually move the needle, that are going to help us understand fit with their culture.