Joe Hockey's resignation from federal parliament opened up the seat of North Sydney for a new candidate. It's a plum Liberal seat. Hockey held it with a margin of 16 per cent, making it one of the party's safest. North Sydney encompasses the municipalities of Lane Cove, Hunters Hill, and part of the Willoughby, as well as the thriving North Sydney business district, where Hockey had his electoral office. It has the second-biggest proportion of high income earners of any of Australia's 150 federal seats and the second-highest proportion of tertiary-educated people, too. It's such a Liberal stronghold that Labor has conceded defeat; it's not even going to put up a candidate for the North Sydney by- election. In short, it's a great opportunity for the Liberals to put forward an outstanding Australian, someone of real accomplishment, a future leader. Instead, the power aristocracy that controls the NSW Liberal party head office declared an "emergency", shut out branch members from voting, and imposed a party machine operative on the electorate.

Unless you chance to follow the inner workings of the NSW Liberal party, you've probably never heard of him, a man of scant accomplishment, Trent Zimmerman. If you want to look him up to see for yourself, don't bother looking for his Wikipedia page. He once had one but it was taken down. A senior federal Liberal describes his achievements with this euphemism: "He's a man of wide experience within the party." The advent of a smiling Turnbull as prime minister glosses over the underlying rot in the foundations of his own party in his home state. The problem isn't that he's particularly dreadful. The problem is that he's not particularly compelling. It's no disgrace that he's the Liberal candidate, only that the party didn't want to look for anyone better. And the biggest problem is that the local branch members were denied a say.

For the record, while it existed Zimmerman's former Wikipedia page said that he is "an Australian political party official and has worked in local, State and federal government. He is a vice president of the NSW branch of the Liberal Party of Australia and deputy chief executive of the Tourism and Transport Forum of Australia," a lobby group for some 200 companies in those industries. He is a former Hockey staffer and twice North Sydney local councillor. What is didn't mention was his real credential. Zimmerman is a close ally of the power behind the NSW machine, Michael Photios, who operates without official title yet controls the dominant faction from the shadows. He was a minister in the John Fahey Liberal government of NSW in the mid-1990s. He was formerly both a member of the NSW Liberal executive, and simultaneously a lobbyist. Early in his prime ministership, Tony Abbott took aim at Photios and those with a similar set-up. He banned them from lobbying his government. Abbott said in September 2013 he was "determined to ensure that you can either be a powerbroker or a lobbyist, but you can't be both". Photios might have had his feelings a little hurt. The firm that he owned part of had donated almost $100,000 to the Liberal and National parties in that very year. Faced with that ultimatum, Photios quit the Liberal executive in favour of his lobbying business. He has since operated at arm's length. His faction is known as the "moderates" but it's a system based on patronage, not ideology.

Do you remember the recent news that the newly minted prime minister, Malcolm Turnbull, had been laughed at when he told the NSW Liberal Council meeting that the Liberal party was not "run by factions"? That's because it is. The NSW division is notorious as the most factionally overheated in the country. It was precisely because of such undemocratic stitch-ups in the NSW division that an eminent Liberal panel, led by John Howard, recommended that the party reform its processes. He urged the party to allow a greater say for the grassroots members. Howard's reforms have had the support of Tony Abbott and the NSW Premier, Mike Baird. The Zimmerman coup is a blatant rejection of the advice of the Howard panel. A prominent Liberal reform activist in the North Sydney electorate, mortgage broker John Ruddick, described it as a case where "the machine gets 51 per cent of the power and bulldozes to 100". Ruddick made headlines when he was twice threatened with suspension from the party for daring to publicly urge reform. Now he's gone further and quit the party in despair.

He's so incensed by the Zimmerman fix-up that he's thinking of standing against him as a Liberal independent in the by-election. "Labor's not running, so this isn't going to be a referendum on Malcolm Turnbull. This isn't going to change the balance of power in Canberra. "If I run I'd say, 'This is a referendum about whether we want a democratised, reinvigorated Liberal party or whether we want the head office machine and its dull candidate to have ever-growing power.'" He says that, if elected to federal parliament, he would vote as a pro-Liberal independent, "I wouldn't do an Oakeshott or a Windsor." And he'd prefer not to have to run at all. He's casting about for another candidate to stand as an independent Liberal against Zimmerman. Although Ruddick was part of the minority faction, the Right, he says his motives are not factional: "I want to blow up the factions." "The Left ran the party in NSW and it was a disaster, the Right ran the party in NSW and it was a disaster, we need a new way, and that is democratic reform. If we have a democratic party, there simply won't be this bikie gang style of factions because you'll have 1000 people voting instead of a handful."

The whole point of reform, Ruddick says, is about attracting candidates of real accomplishment "people like Malcolm Turnbull", not faceless apparatchiks. The NSW Liberal executive has agreed to a tokenistic pilot scheme to allow the local party members to choose their candidates for parliament, but only for six preselections, three federal and three State, over the next five years. A member of the Howard panel in support of democratisation despairs of the chances of real change: "Even if we introduced full plebiscites for all preselections, the factions will end up running them the same way they run the existing system. When you have 1000 members it makes it harder, but if the factions want to run it, they do." The advent of a smiling Turnbull as prime minister glosses over the underlying rot in the foundations of his own party in his home State, but does nothing to improve them. Turnbull's leadership coup was carried out with the strong support of the Photios-Zimmerman "moderates". You won't find him raising a hand against them. This week also saw new evidence of Labor self-preserving aristocracy. The Labor party's Faustian bargain with the trade union movement, slowly strangling the Labor party as a responsible governing party, isn't working for the union movement either.

It's bad enough that the Labor leader's past as secretary of the Australian Workers Union is coming back to haunt him in the Royal Commission on trade union corruption. Worse was Bill Shorten's misjudgement in the present. He allowed himself to be enlisted by two unions, the Construction, Forestry, Mining and Energy Union (CFMEU) and the Electrical Trades Union (ETU) into a scare campaign against the China-Australia Free Trade Agreement. It was a foolish trap for him to fall into. It was a case study in narrow, sectional politics at the expense of the party's broader credibility and electability. Shorten was quickly isolated from his own party, with former Labor leaders and current State premiers publicly distancing themselves from him. Modern Labor under prime minister Bob Hawke, Treasurer Paul Keating and ACTU secretary Bill Kelty led Australia into its era of economic prosperity. They did it, in part, by opening the economy to the world. Australia in the last quarter-century has become a poster child globally for the benefits of an open, competitive economy. Shorten found himself stuck in a campaign against it. Internal pressure built. When Shorten had had enough pain and signalled to the then prime minister, Tony Abbott, that he was ready to negotiate over their differences, he was rebuffed. Abbott preferred to torment and taunt the Labor leader over the deal rather than get it through the parliament.

Shorten got lucky on two counts. He was rescued from his own misjudgement by the quiet, patient diligence of his trade spokesperson, the capable Penny Wong, who worked out a compromise deal. And by the sheer luck that the Liberal party replaced Abbott with Turnbull, who was ready for a deal. Labor will remain prone to capture by the unions so long as it allows its structural embrace to remain. Half of all federal MPs and senators have previously held a paid position in the trade union movement. This is a proportion that chances to align exactly with the unions' share of delegates to Labor's national conference. This week the Australian Bureau of Statistics reported that trade union membership continues to decline. Overall trade union membership fell from 17 per cent of the workforce to 15, a record low. In the private sector it's just 11 per cent. "Australian unions" wrote a former ACTU assistant secretary, Tim Lyons, "have only a few years to change or die". Will they take the Labor party with them? Australia is a democracy; the biggest political organisations that take turns running it are not.

And it shows. Peter Hartcher is the political editor.