Open Sourcing AutoTriageBot

Automatic Verification of Vulnerability Reports

This summer I interned at Salesforce and built a tool that helps sort through an incoming deluge of vulnerability reports on HackerOne. The HackerOne service provides a bug bounty platform that connects vulnerability researchers to companies and manages the process of submitting, resolving, and paying out reports. As of 2017, 30% of all bug reports through the HackerOne platform are not legitimate vulnerabilities. Sorting through an incoming deluge of vulnerability reports can be very challenging and that is why we built and open sourced AutoTriageBot.

AutoTriageBot is a chatbot for the HackerOne platform that can automatically verify, deduplicate, and suggest payouts for incoming vulnerability reports. All of this happens in real-time whenever a vulnerability report is received, leading to faster response times. In our trial testing, we were able to use AutoTriageBot to automatically verify a third of incoming vulnerability reports. Currently it supports verifying XSS, Open Redirect, and SQLi vulnerabilities but it is built in a modular manner so it can be easily expanded.

Let’s take a look at how it works!

The bot kicks off when a vulnerability report comes into via HackerOne:

The bot receives this report and detects that it is about a cross site scripting vulnerability. It attempts to find any duplicate reports and posts an internal comment with information about the reports:

In addition, it posts directions on how to engage with the bot for automatic verification:

The user can then reply with the requested information:

Which AutoTriageBot then verifies:

And suggests a bounty based off of historical data:

How it Works

AutoTriageBot is built on top of Docker Swarm, Selenium, and Python.

All vulnerability tests are executed inside of a single use Docker container. This ensures every test is fully reproducible and fully isolated from all other tests. The tests themselves are done via Selenium in a headless browser, so anything that works in a normal browser, works in our test environment. In addition, the tests can be run in multiple independent browsers so we can even verify vulnerabilities that apply to only a single browser. All of this is transparently done in the background and all that is shared with the user is the result of each test in each browser.

Since AutoTriageBot has to integrate with the HackerOne API, we must securely manage the HackerOne API keys. The API keys are encrypted and stored in Docker’s secret manager. The keys are only shared with a single container that implements a limited and secure wrapper around HackerOne’s API.

Expanding AutoTriageBot

AutoTriageBot is built in a modular manner so that it can be easily expanded to verify other classes of vulnerabilities. These modules are in AutoTriageBot/modules and must adhere to a simple interface described in docs/Modules.md. We’d love to see the modules expanded to include other classes of vulnerabilities such as CSRF, SSRF, or RCE. The code is on GitHub and pull requests are welcome!

AutoTriageBots — roll out!