EMPLOYEES

Here we will discuss the different archetypes you will hire into a Product Security program. These are archetypes that rarely fit perfectly, and the better the candidate, the more of these roles they’ll be likely to fill.

LEADERSHIP

Leadership pushes security into the mainline culture, which is reflected in your codebase. Security bugs are not found unless actively sought out — it is hard work to institutionalize this type of scrutiny. A crucial aspect of leadership is to keep the shame game non-existent and keep product engineers coming to security for sage advice. Respect and reputation are crucial to staying connected with product teams.

Lastly, Leadership becomes the historians of the team and understands the big mistakes made in the past, and use these lessons to promote defenses in the future. When the team goes into incident response mode this role should be the calm champion that keeps things cool.

Recruiting: Strict leadership skills here are just as important in this role as any other. Being a team that risks exclusion by engineers, you need to plan for someone who can bring disparate technical teams together well. Technical mentorship potential is good, but that can come from non-leadership roles too. The leader who acts as a hyper-connected, culture establishing figure in the company is more valuable, especially if there are not well established norms around security.

CONSULTANT

Holding impressive communication skills, this role is proactively invited to join discussion by product teams. They can appear very similar to your leadership roles in skillset, except useful on a project level as opposed to driving the whole culture. They’re able to give very well accepted critical feedback and are well versed in explaining vulnerability nuances and approaches to a fix. Disparate engineering teams will include them early in their development process, and the Consultant can easily make their way into projects that forgot to involve security. They are a “face” for the team when security needs to make a good impression across the organization. This shouldn’t be undervalued, especially as companies become larger and more political.

Recruiting: Strong employees like this are common among those who have done security in short-term consultant gigs for years on end, interacting with all types of negative personalities on audit findings. This becomes a tradeoff, meaning they may be weaker with skillsets to own the follow up projects over a long period of time, or might have inexperience implementing the solutions they’ll frequently recommend. A good consultant understands product goals, and can find compromise that ultimately mitigates risks without going overboard. Do not fill out the team with this role, keep it a minority ratio.

BUILDER

The true software engineer with long term plans to eliminate risk. The type who can build a Brakeman, or create CSP, or an immune system. They could fix one off bugs, but they shouldn’t. They are better suited to work on systemic vulnerabilities. For instance, if product engineers are working with disparate authentication platforms (inconsistent or useless logs, password storage, protocols)— unifying them underneath. Big projects and big wins.

Recruiting: This sort of approach requires a very strategic thinker to tackle the highest value mitigations over time. If easily distracted, they’ll never pay off. Seek out a track record of shipping complex code completed over the long term, where big problems are targeted. Security experience is actually pretty irrelevant here if they can lean on their team, but of course is certainly helpful. Hire this until it’s a majority role.

BREAKER

An adversarial mind who is optimized to violate your expectations of security. Can show you if something is resilient to an attack or not. These sorts of roles frequently trade off their efforts to develop long term defenses for a breadth of knowledge of attack techniques. You’ll ask “is this secure?” to this person frequently.

The most entry level breaker may not even be able to write code at all and could still surprise a seasoned software engineer with an exploit. This doesn’t mean you should hire them — sustainable ties with an engineering organization are better held by comparable engineers.

Note: Some companies explicitly punt all “breaking” to external consultants, in an attempt to maintain a stricter engineering culture.

Recruiting: You do not want to hire the magician who impresses you with blackhat tricks and a knowledge of a few exploit tools. You want to find that individual who can thoroughly understand a new and complex system and then scare you with its potential. They should wield their knowledge to discover unpredictable and risky outcomes. Bug bounty programs have yielded some amazing candidates in my experience when you find a culture fit.

FIXER

This engineer lives for troubleshooting critical bugs, owning the commit, and shepherding the short term fix. They value their volume of commits. They’re all over the codebase and become that broad knowledge base of how each bit of infrastructure works with the next. Of the good engineers I’ve known in roles like this, they’re pretty unhappy with the state of code they work with and draw their motivation from this lack of satisfaction.

The answer here is this role should be encouraged into every engineer on every team.

Recruiting: Hiring dedicated ‘fixer’ role is not worth it unless the person is useful in other categories as well — otherwise they’ll burn out. It’s irresponsible to hire exclusively for this role, a desire (or symptom) of not wanting to institutionalize “fixing” as it relates to security and only respond to the frequent issues that come up. Usually this role emerges naturally in an engineering org, it’s rarely sought out.

SPECIALIST

You may have a few extra-significant risk areas. It may be privacy from significant amounts of user content. Maybe it’s crypto, if you’re a bitcoin company. Malware research, or marketplace abuse, or credit card fraud. Having a couple people serve as in house authorities on a specialized risk will be useful. I’m sure Uber was happy to get the Jeep hackers for this specific reason.

Recruiting: It may be tough to interview for a specialist role if you can’t call them out on bullshit. They’ll know more than you on the subject matter, by definition. Consider bringing on a consultant to sit in on an interview loop. They will still need to be a culture fit and be evaluated on potential output, despite their specialization. Unless there is an incredible volume of work for them, they still need to fulfill other roles. I am very cautious when hiring for a single specialist role in Product Security.

PROGRAM MANAGER

As the team grows you’ll discover the burdens of scheduling external consultant reviews with disparate products (and their follow up mitigations), dealing with vendor checklists, Sometimes talking to potential customer security teams, and managing the overall success of various programs. A rockstar PM will be able to shepherd the construction of high impact frameworks or complex refactors with disparate engineers.

Recruiting: Very traditional PM role, with the exception that you’ll want someone who has mapped security lingo to urgency so they can understand the priority of issues that come up. The various programs Product Security will run will have valuable metrics on their success, insight into where vulnerability is being created, and confidence into which investments have been reduced the greatest risks. Find someone who is data driven and can shepherd engineering projects. Don’t hire this until there are programs to manage.

CHAMPIONS

You may have a disproportionate ratio of mainstream product engineers compared to folks working on Product Security, as far as the organization goes. However, you’ll find as you beef up security consciousness all around, champions around the company will come out of the woodwork to help out. They’ll work on other teams, and that feeds collaboration. You want these allies anywhere you can get them, and they’re an intangible that makes all the difference.

Recruiting: This isn’t really a hiring exercise, but more of creating an approachable group and pushing a mission. Then help will arrive!