London (CNN) Brexit has been a massive headache ever since the UK voted to leave in 2016. It took two prime ministers, 1274 days, three deadline extensions and two general elections for an exit deal finally to be deemed acceptable by the British Parliament.

But it also sucked up oxygen in Brussels, as the EU's diplomatic energy fixated on the single issue of having a country leave its bloc.

In that time, the EU was forced to pay less attention to other problems among its member states. Problems that present a far greater long-term threat to the European project than Brexit ever could.

For the EU is being undermined by nations within its ranks ignoring the rule of EU law, deviating from Europe's high standards on human rights and laughing in the face of freedom of expression.

The most recent example of this comes from Poland, where the country's Supreme Court had to warn the governing Law and Justice party that its proposed judicial reforms could violate European law so blatantly that it might be booted out of the EU.

The court's words might be a little dramatic. The proposed reforms, which would allow the government to punish judges for engaging in political activity, ignore the EU's requirement that courts act independently of government. But that doesn't mean Poland is going to get kicked out of the EU.

First, you cannot officially expel an EU member state. It's possible to suspend a nation's voting rights under Article 7 of the treaty of the European Union, designed to punish nations that disobey the EU's founding principles. But they are officially still a member state. It would require unanimous agreement among the other member states to even have a vote on doing so. And no one who understands EU politics thinks there is any chance of this happening.

"Article 7 was never designed to deal with a situation where there was more than one delinquent state," says Ronan McCrea, professor of European Law at University College London.

Right now, there are several delinquent states causing havoc in Brussels. About 340 miles south of Warsaw, Hungary's Prime Minister Viktor Orban has spent the past decade presiding over assaults on his nation's courts, academic institutions, central bank and press.

The EU has triggered Article 7 procedures against both Hungary and Poland , but both moves led nowhere.

In Malta, Prime Minister Joseph Muscat has been hounded with calls to resign for his government's alleged involvement in the murder of Daphne Caruana Galizia , a journalist who was investigating political corruption. Earlier this week, Members of the European Parliament voted by 581 to 26 in favor of a resolution to start the Article 7 process on Malta. Muscat says he will leave office next month , but denies all allegations of wrongdoing.

JUST WATCHED Fmr. 10 Downing St aide: 'There's a prospect of healing' Replay More Videos ... MUST WATCH Fmr. 10 Downing St aide: 'There's a prospect of healing' 07:37

These are merely the most egregious examples of member states undermining the EU's core principles.

Croatia's government is under pressure for failing to reform existing laws enough to protect journalists from facing legal suits for doing their job. There are similar criticisms of tight press control in Greece and Bulgaria. Bluntly, the old continent is hurtling towards a crisis in mutual trust on values and law. And trust is arguably the central pillar of European unity and stability.

Agata Gostyńska-Jakubowska, a senior research fellow at the Centre for European Reform, explains that the "backtracking on the rule of law in any member states" creates "a challenge to the whole mutual trust. That is a founding principle for crucial projects such as the single market or justice."

The problem is that members of the EU are overseen by the European Court of Justice. National courts are expected to respect European law. McCrea explains that "the web of rules under which members states automatically recognize each others' decisions is threatened by undermining the rule of law. The EU is a very small bureaucracy. It largely depends on national judges and national civil servants to implement the law."

With so many European nations terrified at the prospect of the EU juggernaut taking a closer look at their alleged indiscretions, there is no way that as a bloc, the member states would give a green light to Brussels singling out one member. So Article 7 is a non-starter.

JUST WATCHED Labour party leader refuses to leave after huge defeat Replay More Videos ... MUST WATCH Labour party leader refuses to leave after huge defeat 01:20

All of which backs the EU into a hellish corner. "What might bring the EU to an end? It won't be a meeting of EU member states, saying let's end this. But what might start its collapse is some states saying, we don't really accept the primacy of EU law anymore," says Charles Crawford, a former British ambassador to Poland. "Once nation states start challenging the legal order, that is desperately serious for the EU."

The paradox at the heart of the EU is that it insists all states play by the same rules while also being reluctant to interfere in the domestic politics of any members. In some respects, the EU is little more than an agreement between 28 countries to not cheat the system.

As Europe enters 2020, it needs to address problems it has been ignoring since 2016. Brexit required that all the remaining 27 EU members were firmly on the same page. Brussels succeeded, but in doing so had to play nice with all parties.

"The big cost of Brexit for the EU is that it has distracted time, attention and political capital from addressing the real substantive challenges that the EU itself faces," says Mujtaba Rahman, managing director of the political consultancy Eurasia Group. "Brexit was a team-building exercise where the EU could demonstrate how united they were. But it was really something of a cover to disguise how little they agree on the bigger challenges facing the continent."

Those challenges are not limited to member states undermining EU law. The continent is badly divided on how it should address the bloc's economic challenges and what role Europe should play on the global stage.

"The only leader that has a coherent vision is (French President Emmanuel) Macron," says Rahman. "But he's very divisive in the way he goes about it. It's actually highly counterproductive. He wants to build consensus on these questions but the way in which he talks to other leaders actually hurts." And much as it might hurt the French President to admit, the EU is not banking on his re-election in 2022.

All of which means that the EU enters a new decade staring down the barrel of more Brexit negotiations, huge problems among its members and no coherent plan to get everyone on the same page.

Europe needs to act, if the project is to survive. The question is, should it use a stick or carrot to restore unity? Article 7 might be a non-starter, but the EU can tie how big a chunk of the budget a member state gets to its behavior. And nations like Poland and Hungary need that money. How willing is the EU to shake up the gentlemen's agreement and start interfering with domestic politics?

"This is much more of a threat to the EU than Brexit. Brexit, if anything strengthened the union," says McCrea. Without Brexit to hide behind, the fragility of the union will become impossible to ignore. And right now, there is no obvious path Brussels can take that doesn't risk making everything worse. As 2020 rolls on, Europe's biggest cheeses might come to miss talking exclusively about Brexit.