Here's a hypothesis: Representative democracy is what's happening. Unfortunately, democracy is broken. There's a hidden failure mode, we've landed in it, and we probably won't be able to vote ourselves out of it.

For a while I've had the unwelcome feeling that we're living under occupation by Martian invaders . (Not just here in the UK, but everyone, everywhere on the planet.) Something has gone wrong with our political processes, on a global scale. But what? It's obviously subtle — we haven't been on the receiving end of a bunch of jack-booted fascists or their communist equivalents organizing putsches. But we've somehow slid into a developed-world global-scale quasi-police state, with drone strikes and extraordinary rendition and unquestioned but insane austerity policies being rammed down our throats, government services being outsourced, peaceful protesters being pepper-sprayed, tased, or even killed, police spying on political dissidents becoming normal, and so on. What's happening?

Representative democratic government is theoretically supposed to deliver certain benefits:

Firstly, it legitimizes principled, peaceful opposition within the constitutional framework; we have multiple parties, and the party in power doesn't simply round up the opposition and have them thrown in a GULAG. They concede that the opposition may disagree with the party in power on precisely how the state must operate, but agree that it should operate: the difference is a civilized argument over details, not a knife-fight with totalitarian enemies.

Secondly, it provides for an organized, peaceful succession mechanism. When a governing faction becomes unpopular, it can be voted out of office, and will go peacefully, knowing that eventually their successors will become unpopular in turn, and there'll be another chance to take a bite of the apple. (Totalitarian governments tend to hang on until people start shooting at them, with a variety results we've recently had a refresher course in — Libya, Syria, Egypt, Iran.)

But. But.

What if the channels through which concerned people of goodwill who want to make things better enter the political process and run for election are fundamentally flawed?

Our representative systems almost all run on a party system; even pure PR systems like that of Israel rely on a party list. (I could take out Israeli citizenship and run for the Knesset, but I'd be running as "the Charlie Stross Party", not as myself: if I was a runaway success I'd need to find some extra representatives to tag along on my coat-tails.) Parties are bureaucratic institutions with the usual power dynamic of self-preservation, as per Michels's iron law of oligarchy: the purpose of the organization is to (a) continue to exist, and (b) to gain and hold power. We can see this in Scotland with the SNP (Scottish National Party) — originally founded with the goal of obtaining independence for Scotland and then disbanding, the disbanding bit is now nowhere to be seen in their constitution.

Per Michels, political parties have an unspoken survival drive. And they act as filters on the pool of available candidates. You can't easily run for election — especially at national level — unless you get a party's support, with the activists and election agents and assistance and funding that goes with it. (Or you can, but you then have to build your own machinery.) Existing incumbent representatives have an incentive to weed out potential candidates who are loose cannons and might jeopardize their ability to win re-election and maintain a career. Parties therefore tend to be self-stabilizing.

A secondary issue is that professionals will cream amateurs in any competition held on a level playing field. And this is true of politics as much as any other field of human competition. The US House of Representatives is overwhelmingly dominated by folks with law degrees (and this is not wholly inappropriate, given they're in the job of making laws). The UK's Parliament is slightly less narrowly circumscribed, but nevertheless there's a career path right to the top in British politics, and it's visible in all the main parties: you go to a private school then Oxford or Cambridge, participate in student politics (if you're on the left) or debating societies (if you're on the right), take a post as researcher or assistant for an MP or (less commonly) run for a local council office, then run for parliament. There are plenty of people in every democratic constitutional system who have never held a job outside of politics — and why should they? Such a diversion would be a waste of time and energy if your goal is to make a difference on the national stage.

The emergence of a class of political apparatchik in our democracies is almost inevitable. I was particularly struck by this at the CREATe conference, which was launched by a cookie-cutter junior minister from Westminster: aged 33, worked in politics since leaving university, married to another MP, clearly focused on a political career path. She was a liberal democrat, but from her demeanour, speech, and behaviour there was nothing to distinguish her from a conservative, labour, or other front-rank party MP. The senior minister from Holyrood was a little bit less plasticky, slightly more authentic — he had a Glaswegian accent! And was a member of the SNP! — but he was still one of a kind: a neatly-coiffured representative of the administrative senior management class, who could have passed for a CEO or senior bank manager.

So, here's my hypothesis:

Institutional survival pressure within organizations — namely political parties — causes them to systematically ignore or repel candidates for political office who are disinclined to support the status quo or who don't conform to the dominant paradigm in the practice of politics.

The status quo has emerged by consensus between politicians of opposite parties, who have converged on a set of policies that they deem least likely to lose them an election — whether by generating media hostility, corporate/business sector hostility, or by provoking public hostility. In other words, the status quo isn't an explicit ideology, it's the combined set of policies that were historically least likely to rock the boat (for such boat-rocking is evaluated in Bayesian terms — "did this policy get some poor bastard kicked in the nuts at the last election? If so, it's off the table").

The news cycle is dominated by large media organizations and the interests of the corporate sector. While moral panics serve a useful function in alienating or enraging the public against a representative or party who have become inconveniently uncooperative, for the most part a climate of apathetic disengagement is preferred — why get involved when trustworthy, reassuringly beige nobodies can do a safe job of looking after us?

The range of choices available at the democratic buffet table have therefore narrowed until they're indistinguishable. ("You can have Chicken Kiev, Chicken Chasseur, or Chicken Korma." "But I'm vegan!") Indeed, we have about as much choice as citizens in any one-party state used to have.

Protests against the range of choices available have become conflated with protests against the constitutional framework, i.e. dissent has been perceived as subversion/treason.

Occasionally cultural shifts take place: over decades, they sometimes reach a level of popular consensus that, when not opposed by corporate stakeholders, leads to actual change. Marriage equality is a fundamentally socially conservative issue, but reflects the long-term reduction in prejudice against non-heteronormative groups. Nobody (except moral entrepreneurs attempting to build a platform among various reactionary religious institutions) stands to lose money or status by permitting it, so it gets the nod. Decriminalization of drug use, on the other hand, would be catastrophic for the budget of policing organizations and the prison-industrial complex: it might be popular in some circles, but the people who count the money won't let it pass without a fight.

Overall, the nature of the problem seems to be that our representative democratic institutions have been captured by meta-institutions that implement the iron law of oligarchy by systematically reducing the risk of change. They have done so by converging on a common set of policies that do not serve the public interest, but minimize the risk of the parties losing the corporate funding they require in order to achieve re-election. And in so doing, they have broken the "peaceful succession when enough people get pissed off" mechanism that prevents revolutions. If we're lucky, emergent radical parties will break the gridlock (here in the UK that would be the SNP in Scotland, possibly UKIP in England: in the USA it might be the new party that emerges if the rupture between the Republican realists like Karl Rove and the Tea Party radicals finally goes nuclear), but within a political generation (two election terms) it'll be back to oligarchy as usual.

So the future isn't a boot stamping on a human face, forever. It's a person in a beige business outfit advocating beige policies that nobody wants (but nobody can quite articulate a coherent alternative to) with a false mandate obtained by performing rituals of representative democracy that offer as much actual choice as a Stalinist one-party state. And resistance is futile, because if you succeed in overthrowing the beige dictatorship, you will become that which you opposed.

Thoughts?