Organizing public rallies really is rocket science. A successful launch needs careful planning and good software. Especially when highly vocal users are involved!

How well does open source work for such a demanding application with hundreds of simultaneous events and millions of followers? How is information flow optimized and diverse applications integrated? Designing such systems where a small error could leave thousands stranded, isn’t for the faint of heart.

A team of progressive coders, open standards experts and veteran developers from Microsoft and Netscape show how it can be done. Their application integrates open source, cloud based infrastructure and content delivery networks to ensure that millions of people get to the right event at the right location, every time.

Background

A small army of coders behind Bernie Sanders' 2016 campaign cemented the role of volunteer tech in politics. Progressive Coders Network is a volunteer network created to develop open-source tools for the progressive movement. They generated online tools for his campaign, boosting the movement by engaging millions of people to vote, volunteer, and inform themselves about the issues.

Designing for interoperability

Indivisible is a nation-wide collection of groups committed to grassroots advocacy. All of the groups follow principles which were originally shared by Indivisible’s founders online, but work in a decentralized manner. Collecting, organizing and sharing information with such a broad network is a formidable task. Indivisible uses ActionNetwork, a Constituent Relationship Manager (CRM) to create events and communicate with their followers.

Jason Rosenbaum, the CTO of ActionNetwork knows that organizing a campaign depends on seamless integration amongs multiple data sources and software applications. He designed his platform with a range of integration options and support for Open Supporter Data Interface (OSDI). The group is defining an API and data structures to enable interoperability among products for organizations with progressive values.

Josh Cohen is the Founder and Chairman of the OSDI. He’s ideally suited for the role with over twenty years experience in software standards at Microsoft, Netscape and UPS. Cohen has also been on the IETF Working Group for HTTP and served as the Vice Chairman of the Board for DMTF.org. “Each project, no matter how small or large, can easily plug into an ecosystem and individual coders can help build the movement”, said Cohen.

Flexible data integration

Event data from the ActionNetwork is accessed through the OSDI interface using the HAL Browser. HAL provides a set of conventions for expressing hyperlinks in either JSON or XML. It makes APIs explorable and easier to work with as documentation is easily discoverable. APIs that adopt HAL are easily served and consumed using open source libraries available for most major programming languages.

Event extraction, transformation, and loading (ETL) from ActionNetwork and other data sources is performed by python scripts which are periodically called via a Heroku instance. Usually, there are multiple data sources for the events, various campaigns that are working in unison. Redundant and personal information is first removed. It’s then transformed into a consistent format that’s understandable by the mapping software. Rapi Castillo developed these scripts and they’re freely available as open-source resource on github.

The need for speed

Details on approximately 5,000 events is transformed and compressed daily into a gzip file. Gzip is both a file format and software application for file compression and decompression. The file is then pushed to an AWS S3 instance cached by AWS CloudFront, a global content delivery network (CDN) service. It accelerates the delivery of information at minimal cost. This approach slashes the time needed to load maps from about 3 seconds to under 20 milliseconds.

Mapping made easy

Grassroots campaign rely on creativity to supplement their meager budgets. Creativity also comes in handy when you have to integrate diverse systems yet still scale well. Castillo is a true believer in open source from his experience in developing public maps for the Sanders campaign. The development tools include:

- jQuery a cross-platform JavaScript library to simplify HTML client-side scripting

- D3 a JavaScript library for manipulating documents based on data

- Leaflet an open source JavaScript library used to build web mapping applications

Progressive Coders, the group that he founded uses OpenStreetMaps. This community supported, open source mapping software help thousands of volunteers find Indivisible near them.

Progressive Coders has mastered how to apply open source code, OSDI and CDN to progressive activism. Best of all, just like Elon Musk’s Space-X rockets, their applications are reusable.