Dyson Heydon gave a superb judgment explaining how a scrupulous jurist determined to ignore the political context of his inquiry might not think he's biased. He just did a lousy job putting himself in the shoes of the man on the street, writes Paul Karp.

As Dyson Heydon's decision on his own apprehended bias yesterday showed, the problem with getting a judge to put themselves in the position of the fair minded lay observer is that most ordinary folk only believe one side of an argument, but an eminent jurist can use sophistry to prove any point they want.

A lot of people are having a chuckle at Heydon's defence because he relied on not having read the invitation to the Garfield Barwick address attached to an email, which was printed out for him because he doesn't use the electronic contraptions. Old-fashioned? Guilty, as charged, the former High Court judge said.

But once you get over that hiccup his defence is actually very well argued. It was just a legal speech to a group of lawyers. Persons who were not Liberal members could attend. Speaking at a Liberal Party event doesn't necessarily mean the speaker endorses the views of the Liberal Party. The organiser told Heydon the event was only "nominally" under the auspices of a Liberal branch, which means he understood that it wasn't really a Liberal event at all. What is all the fuss about?

The fuss is about the fact a layman and not a legal genius has to be the judge of Heydon's actions - might a fair minded observer think Heydon may not be impartial as a result of agreeing to the address. So it doesn't matter that speaking at a Liberal event doesn't necessarily mean the speaker is biased, nor that Heydon thinks it wouldn't be rational to come to that conclusion. The question is might an ordinary person think that was the case, and the answer to that is yes.

In relation to pulling out of the event, Heydon said:

Sometimes a decision-maker chooses not to do something, not because to do it will give rise to a reasonable apprehension of bias, but because the decision-maker for sensible reasons of risk management and self-preservation wishes to avoid the attacks of the suspicious and the malicious.

This appears to suggest that Heydon thinks only a suspicious or malicious person could say giving the Barwick address was a sign of bias. This is highly doubtable. Unions - "suspicious and malicious" when it comes to the Royal Commission - have been calling it a political witch-hunt for a year and a half. Nobody much paid attention. But after the revelation about the speech, Heydon's credibility was called into question. The reason this story has played for weeks, and why NSW Bar Association publications manager and Liberal Party supporter Chris Winslow emailed the Royal Commission counsel assisting in a flap when he found out about Heydon's speech, is that everyone knows it's not a good look.

Heydon says ordinary judges have given legal lectures to political organisations, without being subject to criticism. But the Trade Union Royal Commission is investigating unions affiliated to the Labor Party. It has questioned two Labor leaders. It has considered loans from unions to the Labor Party as case studies of wrongdoing and conflict of interest. Put simply - in these circumstances, in this sort of inquiry, the bar is higher.

Which brings me to the weakest point of Heydon's judgment. In a moment of massive overreach, Heydon says the unions simply asserted the link between the politics of Labor v Liberal and the work of the Royal Commission. Plenty of union members are not Labor members, he said. Sometimes unions support the Greens or the Katter Party.

It reminded me of the time the High Court was hearing a case about a ban on union donations to political parties. Justice Kenneth Hayne made an extraordinary intervention when he felt counsel was glossing over the fact the ban would disproportionately harm Labor. "Are we to ignore 100 years of history in this country," he thundered, precisely because it was so obvious the labour movement and Labor Party are inextricably linked. At this point of Heydon's decision, that is precisely what he asked the fair minded observer to do - ignore the disproportionate effect an inquiry into unions affiliated with the ALP has on that party.

Heydon gave the Construction Forestry Mining and Energy Union (CFMEU) a serve for saying it might appear the Royal Commission had "doctored" an email. He criticised the Australian Council of Trade Unions for being too trenchant in saying an observer would be in "no doubt" Heydon knew it was a Liberal event. All this proved the unions' supposedly reasonable observer was nothing of the sort, he said.

But it's criticism like this which shows how unions were damned if they do, damned if they don't. If they went too soft in their applications, Heydon could have said there was no case for apprehended bias to answer. If they went too hard, as they did here, that just discredits the unions too, so Heydon said.

The reason Heydon and the Royal Commission might appear biased is that there never seems to be a Goldilocks zone where unions' behaviour is not too hot, not too cold, but just right. Heydon's TURC interim report blasts the CFMEU as too militant, while the line of questioning regarding the AWU suggests the Royal Commission might conclude they're not militant enough. When Julia Gillard answered questions with confidence, Heydon said her denials were "forced" and "excessive", and that she "protested too much".

Dyson Heydon gave a superb judgment explaining how a scrupulous jurist determined to ignore the political context of his inquiry might not think he's biased. He did a lousy job putting himself in the shoes of the man on the street, or person in the pub, or the woman who can read emails. One might say he protested too much.

Paul Karp is an industrial relations journalist at Thomson Reuters. Find him on Twitter @Paul_Karp.