Summary Socorro is the crash ingestion pipeline for Mozilla's products like Firefox. When Firefox crashes, the Breakpad crash reporter asks the user if the user would like to send a crash report. If the user answers "yes!", then the Breakpad crash reporter collects data related to the crash, generates a crash report, and submits that crash report as an HTTP POST to Socorro. Socorro saves the crash report, processes it, and provides an interface for aggregating, searching, and looking at crash reports. October was a busy month! This blog post covers what happened.

Staff in October For the duration of October, I was on my own engineering-wise. One person isn't a critical mass for a project, so I'm dealing with that by doing self-reviews and discussing the finer points of things with a stuffed owl. Socorro and crash ingestion is a huge project and it's been tough. If you want to talk more about that, toss me an email, ping me on IRC, or catch me at the All Hands in December.

Highlights of October October was busy! Lots of work towards getting rid of alembic/sqlalchemy and moving all database management to Postgres. ([bug 1361394]) Move signature data to a Django-managed table, rewrite all signature-bookkeeping consumers and producers. ([bug 1463121]) Move bug association data to a Django-managed table, rewrite all bug association consumers and producers. ([bug 1493768]) Move platform data to a Django-managed table, rewrite platform consumers. ([bug 1498441]) Move crontabber data to Django-managed tables, rewrite crontabber consumers and producers. ([bug 1493687])

Rewrite BetaVersionRule to use Buildhub for version lookups which fixed a bunch of long-standing issues. ([bug 1501780], [bug 1486035], [bug 1295963])

to use Buildhub for version lookups which fixed a bunch of long-standing issues. ([bug 1501780], [bug 1486035], [bug 1295963]) Add a bunch of dataflow documentation.

Split java_stack_trace into a sanitized and raw versions. ([bug 1496599])

into a sanitized and raw versions. ([bug 1496599]) Split moz_crash_reason into a sanitized and raw versions. ([bug 1502477])

into a sanitized and raw versions. ([bug 1502477]) Add webapp tests to our Python 3 test scaffolding. ([bug 1406703])

During flights and while traveling, Lonnen worked on fixes to migrate us to Python 3.

Bunch of signature generation fixes.

Bunch of other bug fixes, PII access requests, and other things. I'm one or two pull requests away from ditching alembic, sqlalchemy, ftpscraper, a bunch of tables, and all our stored procedures. That's a huge chunk of technical debt and covers a lot of bugs that we'll get to close out. After finishing that, the next project is the Python 3 migration. I also spent some time looking at chkimg as part of annotating crashes when the code in memory doesn't match the binary ([bug 1274628]). That's pretty interesting work. I hope to spend some time with a "cupcake" of this problem to get a better feel for performance issues when operationalizing that utility.