July 6, 2012

Every company with public-facing web applications needs a clear security disclosure policy. This policy serves three main purposes. First it tells people who discover a vulnerability how to proceed and what your response will be so they can report problems easily. Secondly it tells your employees exactly how to process and respond to these reports. Most importantly, a clear public security policy lets your users know in advance how you will respond to recently discovered vulnerabilities.

As I discovered last week when notifying Heroku of a vulnerability in their build system, even the most progressive, respected companies don’t always get it 100% right.

No matter how well designed your application, users will find bugs. When (not if) that happens, it is vital that the person who discovers the vulnerability knows how to report it.

Bug reporting is a funnel, just like every other part of your web application. If users get confused or decide you are not worth their time, they will leave the site without converting. In this case your “users” may be curious hackers, security researchers, or even your own customers. It’s your job to make reporting as easy, painless, and rewarding as possible. If you fail, you risk not finding out about the vulnerability, which could endanger your users’ data and the company’s future. Remember, the person reporting a security flaw is being friendly and doing you a big favor. Treat them accordingly.

Here’s how:

Security Page

At some point, someone will need to report a security vulnerability. You don’t want to have that conversation you in public or in the clear. It is also not a problem you want to outsource to Twitter, Facebook, or a forum. The first thing on your security page should be a link to security@yourawesomeapp.com. The second is a PGP public key, which can be used to send you encrypted messages. This page should always be served over HTTPS. Only by encrypting each step in the communication process can the sender be sure the person they are emailing is who they claim to be without being intercepted. If the PGP key is not posted already, a responsible hacker might email a request for one to be posted which wastes valuable time. The person discovering the vulnerability might become unavailable by the time the key is posted, causing further delays.

Policies

The policies listed on this page should be clear, concise, and friendly. The person who discovers a vulnerability is already in a difficult position and it is important they understand your policies.

Indemnification

A person who accidentally discovers a vulnerability may need to experiment further to see how far the problem goes, and even if it is in fact a vulnerability. Often it is not possible to determine the nature of a vulnerability without trying to do something that should not normally be possible or allowed. Unfortunately parts of this process often necessarily run afoul of the law in some jurisdictions, meaning that if the hacker wants to report a problem to a company, s/he needs to admit to participating in a possibly illegal activity. Some companies make this difficult situation worse by threatening legal action outright or if the disclosing hackers refuse to sign a retroactive NDA. Minor penetration is a routine part of security testing and user exploration. Regardless of the legal threats available to your company, using them against users, hackers, or researchers who acted without malicious intent is never advisable.

For these reasons it is vital that you indemnify and hold blameless anyone who penetrates your site, and in the process of exploring or experimenting, extracts a small amount of sensitive data and promptly notifies you then destroys any data collected. Without a clear and binding promise of immunity from future prosecution, those who discover vulnerabilities may not notify you at all. Publishing a clear policy that protects the party disclosing an exploit is the first, and most important step in building trust.

Bounties

Details about a vulnerability are worth a great deal of money. This kind of information is valuable not only to your competitors, but also on growing black and grey markets. Many other parties would pay for the details of a vulnerability in order to exploit it themselves. This means that anyone discovering a vulnerability on your site could sell it easily. Knowledge of a vulnerability will always be worth more to your company than any of these others. However, a hacker asking for money either before or after making a disclosure can make everyone uncomfortable. Avoid this situation entirely by advertising generous bounties ahead of time. Here’s what some companies are currently offering:

Facebook Starting at $500 Mozilla $3,000 flat fee Google $500-$20,000 depending on severity

The free market determines how much a vulnerability in your system is worth. For products with large user bases, the prices can get very high.

The best companies also make their pre-release code or a staging server with dummy data available for testing with even higher rewards. In this case exploits can be found and remedied before users’ data is ever compromised.

Disclosure

Without the threat of full disclosure, responsible disclosure would not work, and vendors would go back to ignoring security vulnerabilities. Bruce Schenier

Do not ever try to get hackers or researchers to take a bounty in exchange for not publishing their discovery (and it is their discovery). Unfortunately some companies offer bounties only in exchange for silence. Don’t attach strings to bounties. Many hackers prize recognition higher than remuneration, and there’s no need to deprive them of both. Similarly, security researchers make their living off of their reputation for discovering holes. A great policy is to offer a reasonable no-strings-attached bounty and then double for coordinated disclosure.

Coordinated Disclosure is when the researcher who discovers a vulnerability notifies the vendor of the product and allows them a reasonable amount of time to fix it before disclosing publicly.

No matter what, remember that someone disclosing a vulnerability to you directly is usually trying to do the right thing.

Timing

Respond very quickly. When the crisis ends, the discussion will be about how your company handled the situation. It is crucial that the relevant team began work immediately instead of waiting until the next day, or for a lawyer’s permission to talk to the person who reported the problem. Any delay will look like either incompetence or a conspiracy to cover up the vulnerability. You can’t afford for customers or the press to think either. Your customers will be busy checking their own data and then decide if they should change providers. The timing of your response should not be its own reason to leave.

Emails to security@yourawesomeapp.com are your company’s highest priority, period. Your lawyer, investors, and mother can all wait. Messages to that address should be forwarded to the highest ranking member of your security/ops/engineering team on call and the CEO. If you don’t have a 24-hour on-call rotation, create a script with Twilio, Tropo, or Adhearsion that wakes up the CEO or founders. Whoever needs to respond to the problem gets woken up and/or called back from vacation. You asked your customers trust you with their data (and often their own customers’ as well). Many of those affected would gladly wake up themselves to fix it if they were able. You have an obligation to respond as quickly as you are physically able.

Employees

Hearing about a vulnerability is a high stress time for your team. Try not to make it worse. They already feel like they screwed up no matter what caused the vulnerability. Don’t have a blame culture–make it clear you’re all on the same team and share a single mission: doing the best you can for your customers.

Your security team needs to be in direct contact immediately with the person who discovered the vulnerability. Any attempts at legal maneuvering will slow down their response and distract the only people who can fix the problem(s). Make it clear that your team’s responsibility is to patch the application, not manage your company’s strategic communications. If developers are free to treat each other as peers without a lawyer leaning over their shoulders, the patch will come sooner.

Customers

Customers need to know about a problem after it has been patched. Public disclosure means you get caught with your pants down. Discovery without disclosure means someone has a backdoor to your users’ lives and businesses. Which do you think they would prefer? Your obligation to your customers comes first, regardless of any embarrassment it might cause. If world governments are not able to keep their embarrassing secrets, it is unlikely you will be able to either (for long). Honest and direct communication builds user loyalty, hiding the failures that affect your users causes far bigger problems.

Outcomes

All vulnerabilities caught in pre-release stage. Vulnerabilities in production code caught and fixed before public disclosure. Vulnerabilities caught and disclosed to users and company simultaneously. Vulnerabilities discovered by malicious party and exploited without company or user knowledge.

Always remember when you are dealing with disclosure of a security issue that things could be much worse–you could not know about it.