In the wake of Mitt Romney's Election Day defeat, frustration with the Romney campaign's get-out-the-vote effort led to reports of an IT meltdown. Project Orca, the system the Romney campaign used to connect to an army of volunteer poll-watchers tracking voter turnout, quickly became a target for conspiracy theorists, even leading to accusations of sabotage by contract developers. How could the Romney campaign spend so much on technology consultants and yet see its vaunted app fail when it was needed the most?

The truth, according to sources familiar with the matter, is that the applications powering Orca were developed by an internal "skunkworks," one made up of both campaign staffers and volunteers—not by a big-name consulting firm.

Into the skunkworks

According to Federal Election Commission filings that we examined, the Romney campaign spent over $40 million on technology vendors. Targeted Victory, the firm founded by Romney's Director of Digital Zac Moffatt, was the biggest single beneficiary of that spending, receiving more than $14 million as of the end of October. That money was both payment for its own social media consulting and development work, and for "pass-through" IT spending that went to other contractors such as Rockfish Interactive (the developers of Romney's running mate announcement mobile app).

The investment in Orca, however, was a tiny fraction of the campaign's IT spending. Sources tell us the Romney campaign tried systems similar to Orca during the Republican primaries, but on a much smaller scale. To prepare for the general election, the campaign pulled together a makeshift team of IT people and volunteers, and it was this team that built the full version of the Orca Web app and developed the backend systems to power it.

A number of small consulting companies assisted with implementing Orca after its development—specifically, they helped to support more than 800 volunteers who used the system at Romney's headquarters in Boston Garden on Election Day.

Here's how the system was supposed to work—though it probably wasn't a great idea for Romney's communications director, Gail Gitcho, to tell PBS how amazing Orca was going to be before it had been used. "The Obama campaign likes to brag about their ground operation," Gitcho said, "but it's nothing compared to this."

The interface for Orca was a mobile Web application connected to a set of mirrored databases of voter rolls. When a user logged in, the app would load a page with a scrollable list of all the registered voters in the precinct they were assigned to. By swiping a checkbox next to a voter's name, they could record them as having voted; the entry was then transmitted back to Orca's application server. A back-up call response system—similar in nature to the Houdini system used by the Obama campaign in 2008 (a system which also ran into trouble)—provided a way for volunteers to report possible voting irregularities into the system even if their data connectivity failed. Using information gathered through the campaign's digital outreach and pulled from the campaign's voter contact vendor, FLS Connect, the system was supposed to give volunteers in Boston a complete view of Romney supporters in swing states who hadn't yet voted. It would then prompt phone calls asking supporters to vote.

Where were Orca's backend servers located? That remains unclear, though they don't appear to have run in the cloud. Most of the Romney organization's core IT operations, such as desktop support and back-office systems, were outsourced to Best Buy subsidiary MindShift Technologies (a Waltham, Massachusetts managed service provider). But a MindShift spokesperson declined to comment on whether the company's data center was used to support Project Orca, citing a confidentiality agreement with the Romney campaign.

The system had all sorts of problems when it was finally rolled out on the morning of Election Day, but it appears the real issues weren't with the Orca code itself or with its servers. Instead, the problems were in execution. Bandwidth constraints, bad passwords and PINS, and a lag in the operation caused by the surge of initial downloads of data to smartphone browsers all caused widely reported frustrations among volunteers. (Having Comcast suspect the spike in traffic on the lines into the Romney campaign was a denial of service attack didn't help, either.)

Ars was unable to reach Romney campaign Technology Director Kevin Rewkowski or Director of Voter Contact Dan Centinello (Centinello was the political manager of the Orca project) for comment. But there's plenty of reason for them to want to keep a low profile—a website using Centinello's name now accuses him of losing the campaign for Romney with Orca.