When Libra launched on June 18th, it seemed like an alarming new front in Facebook’s megalomaniacal expansion. Having captured billions of users and tens of billions of dollars in annual profits, the company would now be taking over currency itself. The company’s head of blockchain, David Marcus, laid out his plan for Libra in a detailed white paper, with some of the financial world’s most powerful companies already signed on to help govern the new currency as part of the Libra Association. It was Facebook’s vision for an international currency, and based on the company’s partners, it seemed unstoppable.

That was then. The first to ditch Libra was Paypal, which withdrew on October 4th. Then, over the course of a few hours on October 11th, Visa, Mastercard, Stripe and Mercado Pago all bailed on the project, with eBay tagging along for good measure. That meant every major US payment processor has exited the association. (The final remaining payment processor, PayU, has not responded to multiple requests for comment.) It’s an alarming turnaround for the Facebook-backed project, and the first clear indication that Libra’s founders may have bitten off more than they can chew.

This group of companies had particular reason to get cold feet

Losing five companies in the span of a couple hours might seem like a panicked rush for the door, but the timing matters. On October 14th, all the founding members are set to convene in Geneva for the first ever Libra Council meeting. That’s where they will hammer out the different roles to be played by the different parties and try to answer all the governance questions that aren’t spelled out in the initial white paper. Ultimately, that will result in a formal charter, with each member signing their name to the new agreement.

That process is going to involve a lot of specific commitments from specific parties — so if anyone had doubts, the best time to get out was before the charter was signed. It also means that the quick exodus isn’t quite as catastrophic as it looks. Anyone who’s willing to sign their name on the charter is probably in for the long haul.

This group of companies had particular reason to get cold feet. With the exception of eBay, they’re all payment processors, which means they have specific regulatory requirements dealing with fraud, money laundering and sanctions enforcement. Governments were starting to realize that Libra might make it hard to meet those requirements — and payment processors in particular would end up on the hook.

As Senators Schatz and Brown wrote to three of the departing companies, “Facebook appears to want the benefits of engaging in financial activities without the responsibility of being regulated as a financial services company.” The implication was clear: if Libra made things too easy for terrorists and money launderers, Visa, Mastercard and the others might end up taking the blame.

the troubles with payment processors point to a central problem in the Libra project

That’s a scary thought for payment processors, who could see their central businesses wiped out by regulatory action. But for other companies in the Libra Association, that isn’t a problem. The initial coalition includes a range of organizations, including charities, venture capital firms, and service providers like Lyft and Uber. Those companies don’t face the same pressure from regulators, so there isn’t the same benefit to ducking out.

Still, the troubles with payment processors point to a central problem in the Libra project, one it may no longer be able to solve. Blockchain-backed coins work best as a self-contained system, but problems often arise in the places where they overlap with conventional banking. In the early days of Bitcoin, transaction processors were often prosecuted for not upholding anti-money laundering requirements — something many of the pioneering companies had never considered. It’s only recently that US regulators have been able to enforce Know Your Customer requirements across wallet providers. The portals in and out of the ecosystem are the most important and legally sensitive parts of any new currency project — and Libra is no different.

In the initial vision, Libra handed those compliance issues off to payment processors, hoping Visa and Mastercard would be able to handle the intricate compliance requirements involved in trading dollars for Libra. But after Friday’s mass exodus, that’s no longer an option, and the same regulatory tangle that scared off the initial group of processors is likely to discourage others too. Anyone who steps into that role will have a huge task ahead of them, convincing governments around the world to clear the way for Libra at a time when public trust in Facebook has never been lower. It’s not the end for Libra, but the project faces a hard path moving forward — and it’s only getting harder.